Posted by George Moore, General Manager, Live Platform Services

In the AtomPub protocol alignment section of last week’s blog posting by Dave Treadwell, he foreshadowed a “few more surprises in this area to be announced at MIX”. I wanted to take this time to more fully paint the picture of our work behind the scenes over the last few months in the context of the announcements at MIX08.

For the first time ever we have a unified protocol and developer tooling story across most of our major storage products from Microsoft:

• On-premises structured storage: SQL Server
• Cloud-based structured services: SQL Server Data Services
• Cloud-based Live storage services: Spaces Photos and Application Data storage

The unified on-the-wire format is Atom, with the unified protocol being AtomPub across all of the above storage products and services. For on-premises access to SQL Server, placing an AtomPub interface on top of your data + business logic is ideal so that you can easily expose that end point to everybody that wants to consume it, whether they are inside or outside your corporate network.

Layered on top of these protocol standards are a set of URI namespace conventions to address scalar values and feed-of-feed hierarchy navigation which also work uniformly across the above storage products and services, regardless of the top level DNS address of the underlying service:

Reserved keywords in the URI conventions start with $ and are used by the higher-level developer tools and libraries to assist the developer by providing stronger typechecks at design and compile time. The last item in the table above – $metadata – is the key to allow autodiscovery of higher level service-specific constructs by any developer tool against any of the Microsoft service endpoints. More on that below.

While it is certainly feasible to interact with any of these services by simply performing HTTP GET, PUT, POST and DELETE operations against any of these URI constructs, the real value comes from the ability to utilize any of the existing Atom and AtomPub libraries on the web to assist with higher level programming. Microsoft supplies an AtomPub lib via the .NET WCF Syndication for exactly this purpose. In the following code example, we show how using such a library can greatly reduce the amount of coding required to open and parse an authenticated Atom feed for Live Spaces Photos:

// Open an authenticated Spaces Photo Atom feed, parse the top-level
// list of "feeds" (photo albums), and display that in a listbox.
void LoadFolders()
{
     WebClient c = new WebClient();
     c.Headers["Authorization"]="DelegatedToken dt=\"" + DelToken + "\"";

     using (Stream s = c.OpenRead(string.Format(Svc, CID) + "/Folders"))
     {
           // WCF Syndication can directly parse Atom feeds:
           var feed = SyndicationFeed.Load(XmlReader.Create(s));
           foreach (SyndicationItem item in feed.Items)
           {
                DropDownList1.Items.Add(item.Title.Text);
           }
     }
}

However the most efficient programming comes from the highest levels of abstraction provided by ADO .NET Data Services (aka “Project Astoria”) which provide LINQ statements for .NET against any of the above on-premises or cloud-based storage endpoints. The code example below works identically to the WCF library example above, but since Astoria can autogenerate specific types for the various constructs in the Live Spaces Photo feed, the equivalent code that a developer must write is quite a bit smaller:

void LoadFolders()
{
       // Strongly-typed photo-specific constructs are autogenerated by the 
       // Astoria tools by directly parsing the $metadata area of the 
       // Photos service
    
       SpacesPhotosService svc = new SpacesPhotoService(Svc, CID, DelToken);
    
       Foreach (Folder f in svc.Folders)
       {
       DropDownList1.Items.Add(f.Name);
       }
}

And finally, layered on top of all these formats, protocols, and tools is a unified synchronization framework based upon the FeedSync extensions to the AtomPub protocol. The FeedSync specification is available under the Creative Commons Attribution License and the Microsoft Open Specification Promise. Also revealed at MIX is the Microsoft Sync Framework which provides an extensible model for online/offline synchronization and replication of any data source across any device on any network topology. Completing the sync picture is “Astoria Offline” which utilizes the Sync Framework to provide direct synchronization against any ADO.NET provider, including the storage services described at the beginning of this post.

It is hard to overestimate the value and utility of having a robust synchronized “mesh” of devices and feeds. As Ray Ozzie described during his MIX08 keynote:

Just imagine the convenience of unified data management, the transparent synchronization of files, folders, documents, and media. The bi-directional synchronization of arbitrary feeds of all kinds across your devices and the Web, a kind of universal file synch.

To summarize, the complete storage and developer tools stack revealed at MIX08 looks like this (described top to bottom). You are free to utilize this stack at any level of abstraction – there are no requirements to use all layers, and you are free to substitute your own developer tools against any layer:

While all of the above code is available for initial use, this stack is not complete – there is more to come at even higher levels of abstraction. I’ll leave you with an additional foreshadowed reference to future announcements in this space, again quoting from Ray Ozzie’s MIX08 keynote:

Before you know it, you in this audience are going to have the option of being the first to try out an early technology preview of this simple but incredibly useful new software and service. As this product emerges just over the horizon, I think you'll find it to be quite intriguing and key in delivering upon a compelling vision of a personal device mesh and of connected devices.

For more information on any of the items in this post, please watch the following MIX08 presentations:

• Microsoft Sync Framework
• Accessing Windows Live Services via AtomPub
• RESTful real world applications using the ADO.NET Services Framework
• Introducing SQL Server Data Services

-- George Moore