Lost In Tangent

I had a dream once where all developers had to care about was server code. We could just write services all day long without any regard to anyone actually using the functionality. It was a little lopsided and ridiculousness, but it made the job half as easy. Well, unfortunately we don’t live in that crazy fantasy world. Now that we’ve created this sweet data service, we have to actually create an application to consume the thing (absolute heresy!). Luckily for us, it’s actually really easy. In fact, I question whether it’s a little too easy. We all know that good code should be hard to read and so convoluted that not even The Da Vinci Code could decipher it. Maybe we can let our guard down just this once, and embrace the slim data service client API.

From previous posts, you’ll recall that because the data service has a RESTful interface, all we really need to work with it is a class that could make HTTP requests and give us the response. Which means if you really wanted to you could use the WebClient class like so:

Note: I truncated the URL parameter passed to DownloadString to conserve space. I’ll be doing a lot of this through the rest of the article.

You could then use LINQ To XML to parse the carriers string, and you would look like a coding god. You just used a class in the System.Net namespace, which is an automatic 100 cool points! Not so much. Treating the data service like a plain HTTP resource makes sense only in the minutest scenarios, and even then I don’t see why you would use the above code. When working with a data service, there are numerous additional features we need to make the whole process cleaner, that WebClient (or HttpWebRequest) can’t give us. I can think of at least four things I’d need out of a data service client:

1. Object deserialization
2. Lazy loading
3. Identity tracking (Identity Map for all you pattern nerds)
4. Change tracking (Unit of Work)

(Note: For anyone keeping track, I just covered all of Martin Fowler’s object-relational behavior patterns)

By “lazy loading”, I mean that we should be able to put off the retrieval of superfluous data until we actually need it. By “identity tracking” I mean that I need something that is smart enough to know what items I’ve already requested from the data service (based on identity), so that I only have to retrieve it once. By “change tracking” I mean that I need to be able to add/edit/delete items in the data service, and those changes (and only those changes) be tracked and persisted. #1 should be pretty self-explanatory. I don’t want to have to get a string of XML and have to parse that trash myself. When I request a Carrier from a data service, I want a Carrier object, not a <carrier> element.

All four of those tenets are satisfied by any worthwhile OR/M framework. LINQ To SQL’s DataContext handles that load, and the Entity Framework’s ObjectContext does too. To keep pure synergy across the entire suite of new ADO.NET tools, ADO.NET Data Services has been given the WebDataContext class. Not only does the name fit ([InsertWordHere]Context), but it contains the necessary functionality for abstracting/working with a data service, as well as providing the aforementioned features.

Creating a WebDataContext is as easy as:

“Where is my connection string!”, you might be saying. With data services, the URL of the service is the connection string. I don’t know why but that thought was a turning point for me in appreciating data services. The clearer it becomes that you’re able to interact with a data source just like you would using familiar/traditional methods (i.e. LINQ To SQL, EF), but over the wire, the more appealing this technology becomes.

So now that you’ve got your context, pointing at the right data service, how do you get data out of it? The WebDataContext class has a method called CreateQuery that takes a string representing the data query: entity name, and query parameters. For instance, if I was using the above context, and wanted to get all cell phones, I could simply do:

If I wanted to get all cell phones that have email, ordered by price, I could do:

Nothing new is going on here, and it isn’t necessarily impressing. At this point all we’ve really added is some semantic meaning to working with our data service that we didn’t have using WebClient or HttpWebRequest. That is definitely a benefit, but it isn’t incentive enough for it to be a save-all solution. We need to align the three tenets above to functionality of the WebDataContext for things to get really juicy.

1) Object Deserialization

This is absolutely the most important tenet (I suppose that’s actually a subjective statement). If you’re already used to using OR/Ms to streamline your data access, then you probably expect the same ease-of-use when working with data services. You may have noticed that the CreateQuery method is generic. It accepts the object type that represents the entity being requested. Then what happens is that the WebDataContext will attempt to serialize instances of the specified type from the data returned from the call to the data service, that way you can work with a collection of objects (or just one, depending on the query).

You may be asking, “Where on earth does that CellPhone class come from? This is the client after all, and that class was part of the LINQ To SQL model on the server.” And that would be a great question to ask. The cool thing is that the WebDataContext can perform the deserialization using pure POCO, which means that all your client application has to have is an object with the same name and same properties as the entity on the server, nothing else (there is an exception to this, that we’ll discuss later). This is great for simple types. For me to make requests for carriers I could simply declare the following type:

And then make the following call:

I now have a collection of Carrier objects, contained by a WebDataQuery object. Beautiful! We’ll discuss the benefits of the WebDataQuery class in the next post, for now it’s sufficient to note that it is just a client container for a data service query.

What is really cool about the above example is that for a client to consume a simple data service, they don’t need a bloated proxy. All you need is the URL of the service, and a plain object that matches that of the consumed entity. That kind of code cleanliness/simplicity makes me very happy

The first thought you might be having is: “I agree this is some great stuff, but what if I want to work with a data service that has multiple types, that are fairly complex? Hand coding those classes could get time-consuming.” You would be exactly right. The above example was meant to illustrate how basic working with a data service can be, but it wouldn’t always be practical to hand-code the client-side model. Luckily, ADO.NET Data Services comes with a tool to generate your client-side model. All you have to do is give it the URL of the data service and it will generate classes for each entity exposed by the service. It will also create a strongly-typed subclass of WebDataContext that provides nice properties for accessing each entity (just like LINQ To SQL and the Entity Framework). The tool is called webdatagen and is located in the Program Files\Microsoft ASP.NET 3.5 Extensions directory (this location is subject to change).

To illustrate it’s use, I’ll leverage webdatagen to create my client-model for the cell phone data service:

/outobjectlayer - This is the name of the file you want the generated classes to go in (the default language is C#, but you can set it to use VB)

/uri - Obviously this is the URL of the data service you want to generate the client model for

The webdatagen tool has a lot more options than this, but I’m not going to go into them. You can type /help to get all that info. The important thing is that after running the above command, I now have a file called CellPhoneModel.cs that contains classes for Carrier, Manufacturer, and CellPhone (those of the entities in that service), as well as a nice WebDataContext. I include the file in my project and can now run the following code:

That’s definitely a nicer alternative to what we were using before. Our code is now more explicit, and we’ve abstracted how we’re accessing the CellPhone data using the newly generated property. If you wanted to take it a step further, you could either create a default constructor on your generated WebDataContext that infers the service URL from somewhere (hard-coded, config file, etc) or you could use an IoC container to inject the URL into the existing constructor. Either way, you can see how easy working with a data service can be.

If you’re curious how the CellPhones property works, here is the generated code:

As you can see it’s the same exact logic we were running manually before, except in this case it’s also being smart enough to “cache” the CellPhone data, which we weren’t doing before.

2) Lazy Loading

Now that we are working with strongly-typed objects, we no doubt want to be able to access both scalar and relational properties, otherwise what’s the point of having a client-side model?. As you may recall from article #3, when you request an entity from a data service, its relationships are not initially loaded, which is a good thing, because you could potentially have a lot of relationships on an entity, that could result in a ton of unneeded data being transfered.

When making a request through a URL in a browser, or using the WebDataContext’s CreateQuery method, you can use the $expand query parameter to eagerly load any relationships. This approach is great, but also has a couple draw backs:

1. You have to know at the time the query is made that you want to expand a relationship (or relationships).
2. If you are querying an entire entity set, and you want to expand a relationship property, you have to expand it for every single entity in the collection. If you only want the relationship expanded for certain entities, based on some type of criteria, there is no way to do that, and you’ll end up transferring a potentially large amount of unneeded data.

Imagine I want to make a query for all cell phones. Easy right? In fact, we already saw that example done very simply in the previous section. Let’s revisit it for the sake of repetition…

That will return to us a collection of CellPhone objects, but their Carrier and Manufacturer properties won’t be populated (thanks to lazy loading). We decide that we also want to receive the cell phone’s manufacturer, so we modify the above code:

Perfect! We now have our manufacturer data. We later figure out that we only need the manufacturer data for phones that are less than $500 and weights more than 3 ounces (that is a ridiculous need, but work with me here). For us to get manufacturer data using the CreateQuery method, it’s all or nothing in regards to retrieving relationship data. We know we need to get some manufacturer data, but we’re really hesitant to just blindly pull down all manufacturers when we only need a few. It’s clear that CreateQuery doesn’t offer the functionality we need.

Luckily, CreateQuery isn’t the only method the WebDataContext class has to offer. LoadProperty allows you to specify an entity object instance, and the name of one of its relationship properties that you want to populate. LoadProperty is great because it’s an “after the fact” method. It allows you to query the base data you know you need up front, and then later pull down just the extra stuff, as you need it. In our current example, we want to query all cell phones (without expanding any relationships), and then programatically determine which cell phones need their manufacturer data, and at that point retrieve it. The following code would work beautifully:

“Isn’t that going to make a bunch of individual calls to the data service to retrieve the manufacturer data? We may have saved bandwidth up front by not expanding manufacturers, but now we’re going to pay the processing price by making a lot of extra calls to the service.” That is very true. The above example wasn’t meant to say that LoadProperty is the best solution every time, nor is making a broad query with CreateQuery the best solution every time. The point is, you have two options for performing your lazy loading, each of which provides a different level of precision with your data. If only 3 out of 1,200 cell phones meet the above criteria, then there is a good chance that using LoadProperty will be more performance. But if 639 out of 1,200 cell phones meet the criteria, you might want to weight the advantages between bandwidth and server calls.

You may be thinking to yourself right now, “You know, I’ve gotten really comfortable with the way LINQ To SQL handles lazy loading of relationship properties. All I have to do is access the property, and behind the scenes it pulls down the data I need. I don’t have to make any extra method calls, it just works. LoadProperty seems like an extra step to me.” You’d be very right. LINQ To SQL (and the Entity Framework for that matter) does make the process of lazy loading a little more streamlined. But, have you ever taken a look at how LINQ To SQL accomplishes its lazy loading? It uses three helper classes (EntityRef<T>, EntitySet<T>, Link<T>) to represent the lazy loaded properties, and relies on code in the property’s getter/setter to work with the underlying helper class reference. It isn’t hard to understand/develop, but it’s added code and added complexity. With ADO.NET Data Services, you don’t need any helper classes, and you don’t need any extra code in your properties. Unfortunately you don’t get the extra behavior, but you do have less dependencies in your entity classes, and less bloat.

You could easily hand-code an ADO.NET Data Services client-model, because it comes very close to being true POCO. The same couldn’t really be said about a LINQ To SQL model, and certainly not for an EDM. I personally love the leanness of the data service client model, and I think a call to LoadProperty is a fair trade-off. In fact, you could even look at a call to LoadProperty as adding a little more explicitness to your code, that you wouldn’t have by just accessing the property and having the plumbing take care of the magic for you.

3) Identity Tracking

So why is identity tracking so important exactly? Well as you can imagine, every time a client requests data from a service, it requires time to return the response, and it requires bandwidth from the server to satisfy the request. In the case of Phillip’s Cell Phone Shack, he is in the business of making money, and having to serve the same cell phone data over and over again, when unnecessary, isn’t the greatest idea ever conceived. If the client was stateless (web application) then such tracking wouldn’t hold as much merit, but if the client was a desktop application, you can imagine that being able to hold a local cache of entity data would save a lot of needless round trips.

Once again, just like LINQ To SQL’s DataContext, and the Entity Framework’s ObjectContext, the WebDataContext class has identity tracking built into it. If you request a cell phone object, and then request the same cell phone again, it will be smart enough to give you the existing instance and not make the service call. This is not groundbreaking functionality by any means, but it’s a requirement for efficient applications, and it’s great that it’s a part of the ADO.NET Data Services feature set.

Above, I mentioned that data services work with pure POCO, and that statement is true as you saw. I also mentioned that there was an exception. In order for the WebDataContext to perform identity tracking, it has to know how to determine the identity of each entity type it manages. In order for it to do that, each entity type has to abide by the same rules as the entity type on the server. The only difference is that instead of applying the WebDataKey attribute to your class, you apply the Key attribute. So if I wanted to enable tracking for my Carrier entity, I could simply do:

If you hand-coded your entity classes, and don’t care about identity tracking, then you’re fine. If you hand-coded your entity classes and you do care about identity tracking, then you’ll need to make sure your entity’s identity property is apparent to the WebDataContext. If you used the webdatagen tool, it assumes you care about identity tracking and applies the Key attribute to the entity type’s identifying property, if it needs to. I say “if it needs to” because as you remember, if the identity property is “ID” or “[EntityType]ID” then that will automatically work, just like the server-side model types. In our case, none of my entity types follow either of those conventions, so I need to explicitly use the Key attribute. If your entity type’s identifying properties do in fact follow one of those conventions, then your data service client could work with pure POCO and leverage identity tracking as well. How sweet is that

So now that your entity classes are identity tracking enabled, there is still one more step before you can actually benefit from identity tracking. Because identity tracking is an additional feature that may not be needed by the client application, it is turned off by default. In order to enable it, you have to set the WebDataContext’s MergeOption property to something other that NoTracking (the default value). The following code would enable identity tracking for our context:

By setting the MergeOption to AppendOnly, you’re telling the WebDataContext to only append new entity instances that it doesn’t already have, effectively turning on identity tracking. The MergeOption enumeration has two other values: OverwriteChanges, and PreserveChanges. These values relate to the change tracking features of the WebDataContext.

4) Change Tracking

The unit of work pattern is great because it allows you to make a bunch of changes in a single session and then submit all those changes for persistence in one swoop. This is by no means new, in fact .NET developers have enjoyed it for years. LINQ To SQL’s DataContext, Entity Framework’s ObjectContext, and even the DataSet, all represent implementation’s of the unit of work pattern. Thankfully, the WebDataContext is no different. It allows us to make many different changes to its entity data (add/update/delete) and then perform a batch save. Best of all it will only persist the data and entities that have been modified, which means no unnecessary data being sent across the wire and no unnecessary logic being performed.

“Woah woah woah!”, you may be saying. “What is all this jibber jabber about changing entity data? Can we even do that with a data service? I thought this sucker was read only? How do you even perform updates to a service that is RESTfully exposed anyways?”. Those are all good points. The cool thing is, yes, you can make changes to a data service’s entity data from a client application. This topic deserves an article in itself, and as such will be tabled until part 7 of this series. Fear not faithful reader, you will wield these great powers very soon. Just be patient and let me finish explaining the entire client-service querying capabilities gamut.

So we’ve added the WebDataContext class to our arsenal of tools. We’ve learned how to leverage the webdatagen tool to build our lean client-side model. We’ve seen how to benefit from lazy loading, identity and change tracking, as well as get full object deserialization. One problem still exists though: a richer coding model for querying the data service seems in order.

If you’ve gotten used to using LINQ, no doubt you might see the above code samples as feeling a little primitive in comparison. Well, today is your lucky day, because there is a new LINQ flavor in town, and it goes by the name LINQ To Data Services. In the next article, we will see how we can leverage our knowledge of consuming data services and throw it away. LINQ is in baby! Everything else is out!

In case someone hasn’t noticed, I’ve been posting a hell of a lot of material lately on ADO.NET Data Services. I’ve created a new top-level page on the right navigation, underneath “Toolbox”, that will aggregate the links to all the articles in the series, and display the topics of upcoming articles. When the series is all said and done, that series page will maintain links to each individual post.

If anyone is going to link to one of the specific articles, I encourage you to link to the series page instead, so that as I post new articles, you won’t miss them.