Lost In Tangent

Phillip gets a call today from a customer:

“Philly my boy, I love the service you setup. This data is going to jump start my website immediately! You know what though, I’ve been trying to get into this AJAX stuff, and as such I was curious whether or not that would work with your service? I also wanted to thank you for dinner last night, I had no idea how sweet you could be.”

Alright, so that call was half weird (alright, a lot weird), but half intriguing. Phillip had to ponder for a minute whether or not his cell phone data could in fact be consumed with JavaScript. He had been spending so much time and effort figuring out how to write client code in .NET (about 2 articles worth), he completely forgot about the possibility of customers wanting their web sites to consume his data client-side.

The ASP.NET AJAX client library does in fact contain the Sys.Net.WebRequest type, and since an ADO.NET Data Service is exposed through a simple REST interface, you could get away with using it to make simple service requests. That seems like a pretty poor option, as we’ve already gotten comfortable writing client code that had semantic value for working with data services (i.e. WebDataContext). Luckily for us we weren’t the only ones thinking that. The ADO.NET team got together with the ASP.NET AJAX team and said “Let’s merge our two worlds together and make something awesome!”

There is a new namespace in the ASP.NET AJAX client library: Sys.Data. This new namespace houses one particular object of interest to us: DataService. “What exactly is DataService and what can it do for me?”, you say. It is pretty much everything you’d need from a lightweight, JavaScript abstraction over a data service. Behind the scenes it’s actually using WebRequest, but at least we don’t have to worry about those details

“How do I get the DataService object into my app?” You simply add a script reference to your ScriptManager like so:

That will add the reference to the ADO.NET Data Service client library, and make the DataService object available for use.

“How did I use the Data Service?” Just like the WebDataContext, the DataService’s constructor takes the URL of the service:

Pretty simple huh? Indeed. “How do you get data out of it?”

Notice a pattern? Consuming a data service from JavaScript is really easy.

The four parameters the DataService’s query method takes are pretty self-explanatory: the data service query, a method that will be called upon success, a method that will be called upon failure, and an arbitrary piece of context data. The failure callback and context data are optional, so if you really trust your query and/or don’t need any context, you could certainly omit them

The query parameter uses the same syntax that we’ve already gone over in previous articles, so there is nothing new going on here. The DataService object calls the specified data service, uses the specified query, and then when complete, will call the success or failure method based on what happened, passing the context data, if provided. It doesn’t get much easier than that.

Ok great, so we make make the above query call, now what? Well we have to implement the actual methods for our success/failure callbacks. The signature for each take three parameters: the query result, the context data (if specified), and the actual query itself.

Our success method looks like so:

In the case of the success method, the result parameter would be the data requested from the service call. It could be in the form of a collection of objects, a single object, or a scalar property value, depending on what your query was. The context parameter would hold whatever was passed to the context parameter in the call to the DataService’s query method. If null was passed, or the parameter was omitted, then the context parameter of the success method would be null. In our case, the context parameter would be the string “Context Data”, because we passed that in when we called service.query(). The operation parameter is the query you made that succeeded. In this case it would be “Carriers?$orderby=Name”.

The context and operation parameters are more for diagnostic purposes, and probably won’t be used all that much, and as such can be omitted from the method signature if you’d like. The result parameter is the primary focus here. As was mentioned previously, the result parameter will contain whatever data it is we requested, and can be categorized into three types:

1. Collection of objects
2. Single object
3. Scalar property value

If your query is targeting an entity set, like we are doing above (Carriers?$filter=Name), then the result parameter will contain an array of objects. The object type will correspond to whatever the service model object is. In this case it would be Carrier objects. Using the above query, I could modify my success method like so (notice I’ve omitted the context and operation parameters to show how that could look):

This would result in an alert message reading “Alltel”. As you can see, I’m accessing the carrier through an index on the result parameter (which in this case is just a simple array). I could just as easily enumerate through the list, and perform whatever actions I needed to with the returned data. The possibilities are endless, and also out of the scope of this article. The point is that you can see how amazingly easy I can write JavaScript code to perform an AJAX call to a data service, and then receive my requested data.

What if I modified my query like so (I’ve omitted the failure callback and context parameters to show how that could look):

In this case, the result parameter passed to the success method wouldn’t be an array of carriers, but instead a single instance of a carrier. I could modify my success method code like so:

Notice that I no longer have to access the carrier via an index on the result parameter, because it’s receiving a single carrier. Nice and clean.

Now what if I modified the query like so?:

In this case, the result parameter passed to the success method wouldn’t be an instance of a carrier, but simply the requested carrier’s name. I could modify my success method code like so:

You might be wondering why I just wasted all that time to illustrate something that probably could have been deduced by someone with half a brain? Because I wanted to make it crystal clear how flexible the DataService object is. When making AJAX calls, you know that bandwidth and processing time are important, so you should be able to request/work with only the data you absolutely need. With the DataService object you aren’t forced to pull down extra bloat, you can streamline your data access as granular as you need.

So what about the failure method? It takes the same three parameters as the success method: result, context, operation. The context and operation parameters are identical to their success method counterparts. The only difference in the failure method over the success method is that the result parameter isn’t the requested data, but rather an object that explains why the query failed.

As I mentioned above, the query method’s failure callback and context parameters are optional. What I didn’t mention is that you can set default values for them on the DataService, which will be used every time you call query and omit those parameters. I could use the following code:

This would obviously fail due to my gross misspelling of the word “Carriers”. Notice I didn’t pass a failure callback to the query method, which would normally mean that the error would get “swallowed”, but because I set a default failure callback, onFailure will be called. Also notice that I omitted the context data parameter, but I set a default context value of “Sweet1″, which will be passed to the call of the onFailure method when this query fails. You can begin to see the value of the context parameter now knowing that you can use a common failure method accross all queries. The context parameter can well, add context to the query that failed.

Aside from the query method not requiring a failure callback or context parameter, you can also omit a success callback. Obviously it wouldn’t make much sense to omit the success callback, since your call to query would become completely pointless. But, just like the failure callback and context data, you can set a default success callback as well, like so:

Notice my call to the query method takes only the actual data query, all the other parameters come from the default set on the DataService. In some situations this would make the code much cleaner, but in others it might encourage some nastyness. The point is to use what makes sense for your scenario. Note, that the query method can still accept all of its parameters, which would then override whatever default values had been set. Hence, you could imagine a nice setup with default callbacks and context, that would only be used for query calls that could be generically handled, while other query calls had their own tailored callbacks.

If you don’t want to let the query call hang forever, you can set a timeout on the DataService, which will kill the query call after the specified amount of time (in milliseconds), and then call the failure method. The following code would set a 1 second timeout on the DataService:

Now every query call made on that instance of the DataService would be limited to a second of processing time.

Remember when I mentioned that the result parameter passed to the failure method contained information about why the query failed? That parameter is actually an object of type Sys.Data.DataServiceError. That object is perfect for determining whether the query failed because of an actual exception, or simply a timeout, since it has a property called timedOut. I could modify the failure method like so:

Also notice the use of the message property, which provides us with the underlying exception message. The DataServiceError also has the following properties for determining the cause of the error: exceptionType, statusCode, and stackTrace. The exceptionType property returns the name of the exception type that actually occurred (i.e. System.OutOfMemoryException). The statusCode property returns the HTTP error that occured (i.e. 404). The stackTrace property returns what it sounds like, the stack trace that led to the query failure.

At this point Phillip feels pretty proud of himself and all that he had discovered about using ADO.NET Data Services together with AJAX. Just as he is about to call his client back to tell him the good news, he realizes something: XMLHttpRequest (and JavaScript for that matter) can’t make cross-domain calls! If his client wants to access his cell phone data via JavaScript, he’ll have to make his own service that interfaces with the cell phone service, then he can use the DataService object to access his local service. Not exactly ideal, but it shouldn’t cause to much pain. With the said, it should be noted that the above samples and newly introduced classes are meant only for accessing data services within the same domain. That constraint shouldn’t really come as a surprise, since we already had to come to grips with that with web services within our application, data services are no different.

You may be thinking “Aha! I see what this is good for! All those times that I created a web service in my web application purely for the sake of calling it using ASP.NET AJAX to get and update data could now be replaced with a data service! That would allow me to create the same user-experience, but without having to write all that data-centric logic myself! Eureka!” You would be a pretty smart person if you said that, and that is exactly the use of the DataService that I would recommend A data service is by no means meant to be exposed over the web and usable by other people’s scripts. It is meant to compliment its parent web application, in the same domain.

The DataService has a couple more methods, but they pertain to change modification, something that we haven’t covered yet. In our next article we’re going to jump back to writing .NET data service client code to go over how we can add/update/delete data from a data service. We’ll also cover how to achieve the same functionality in JavaScript using the client-side DataService object.

I had a dream once where all developers had to care about was server code. We could just write services all day long without any regard to anyone actually using the functionality. It was a little lopsided and ridiculousness, but it made the job half as easy. Well, unfortunately we don’t live in that crazy fantasy world. Now that we’ve created this sweet data service, we have to actually create an application to consume the thing (absolute heresy!). Luckily for us, it’s actually really easy. In fact, I question whether it’s a little too easy. We all know that good code should be hard to read and so convoluted that not even The Da Vinci Code could decipher it. Maybe we can let our guard down just this once, and embrace the slim data service client API.

From previous posts, you’ll recall that because the data service has a RESTful interface, all we really need to work with it is a class that could make HTTP requests and give us the response. Which means if you really wanted to you could use the WebClient class like so:

Note: I truncated the URL parameter passed to DownloadString to conserve space. I’ll be doing a lot of this through the rest of the article.

You could then use LINQ To XML to parse the carriers string, and you would look like a coding god. You just used a class in the System.Net namespace, which is an automatic 100 cool points! Not so much. Treating the data service like a plain HTTP resource makes sense only in the minutest scenarios, and even then I don’t see why you would use the above code. When working with a data service, there are numerous additional features we need to make the whole process cleaner, that WebClient (or HttpWebRequest) can’t give us. I can think of at least four things I’d need out of a data service client:

1. Object deserialization
2. Lazy loading
3. Identity tracking (Identity Map for all you pattern nerds)
4. Change tracking (Unit of Work)

(Note: For anyone keeping track, I just covered all of Martin Fowler’s object-relational behavior patterns)

By “lazy loading”, I mean that we should be able to put off the retrieval of superfluous data until we actually need it. By “identity tracking” I mean that I need something that is smart enough to know what items I’ve already requested from the data service (based on identity), so that I only have to retrieve it once. By “change tracking” I mean that I need to be able to add/edit/delete items in the data service, and those changes (and only those changes) be tracked and persisted. #1 should be pretty self-explanatory. I don’t want to have to get a string of XML and have to parse that trash myself. When I request a Carrier from a data service, I want a Carrier object, not a <carrier> element.

All four of those tenets are satisfied by any worthwhile OR/M framework. LINQ To SQL’s DataContext handles that load, and the Entity Framework’s ObjectContext does too. To keep pure synergy across the entire suite of new ADO.NET tools, ADO.NET Data Services has been given the WebDataContext class. Not only does the name fit ([InsertWordHere]Context), but it contains the necessary functionality for abstracting/working with a data service, as well as providing the aforementioned features.

Creating a WebDataContext is as easy as:

“Where is my connection string!”, you might be saying. With data services, the URL of the service is the connection string. I don’t know why but that thought was a turning point for me in appreciating data services. The clearer it becomes that you’re able to interact with a data source just like you would using familiar/traditional methods (i.e. LINQ To SQL, EF), but over the wire, the more appealing this technology becomes.

So now that you’ve got your context, pointing at the right data service, how do you get data out of it? The WebDataContext class has a method called CreateQuery that takes a string representing the data query: entity name, and query parameters. For instance, if I was using the above context, and wanted to get all cell phones, I could simply do:

If I wanted to get all cell phones that have email, ordered by price, I could do:

Nothing new is going on here, and it isn’t necessarily impressing. At this point all we’ve really added is some semantic meaning to working with our data service that we didn’t have using WebClient or HttpWebRequest. That is definitely a benefit, but it isn’t incentive enough for it to be a save-all solution. We need to align the three tenets above to functionality of the WebDataContext for things to get really juicy.

1) Object Deserialization

This is absolutely the most important tenet (I suppose that’s actually a subjective statement). If you’re already used to using OR/Ms to streamline your data access, then you probably expect the same ease-of-use when working with data services. You may have noticed that the CreateQuery method is generic. It accepts the object type that represents the entity being requested. Then what happens is that the WebDataContext will attempt to serialize instances of the specified type from the data returned from the call to the data service, that way you can work with a collection of objects (or just one, depending on the query).

You may be asking, “Where on earth does that CellPhone class come from? This is the client after all, and that class was part of the LINQ To SQL model on the server.” And that would be a great question to ask. The cool thing is that the WebDataContext can perform the deserialization using pure POCO, which means that all your client application has to have is an object with the same name and same properties as the entity on the server, nothing else (there is an exception to this, that we’ll discuss later). This is great for simple types. For me to make requests for carriers I could simply declare the following type:

And then make the following call:

I now have a collection of Carrier objects, contained by a WebDataQuery object. Beautiful! We’ll discuss the benefits of the WebDataQuery class in the next post, for now it’s sufficient to note that it is just a client container for a data service query.

What is really cool about the above example is that for a client to consume a simple data service, they don’t need a bloated proxy. All you need is the URL of the service, and a plain object that matches that of the consumed entity. That kind of code cleanliness/simplicity makes me very happy

The first thought you might be having is: “I agree this is some great stuff, but what if I want to work with a data service that has multiple types, that are fairly complex? Hand coding those classes could get time-consuming.” You would be exactly right. The above example was meant to illustrate how basic working with a data service can be, but it wouldn’t always be practical to hand-code the client-side model. Luckily, ADO.NET Data Services comes with a tool to generate your client-side model. All you have to do is give it the URL of the data service and it will generate classes for each entity exposed by the service. It will also create a strongly-typed subclass of WebDataContext that provides nice properties for accessing each entity (just like LINQ To SQL and the Entity Framework). The tool is called webdatagen and is located in the Program Files\Microsoft ASP.NET 3.5 Extensions directory (this location is subject to change).

To illustrate it’s use, I’ll leverage webdatagen to create my client-model for the cell phone data service:

/outobjectlayer - This is the name of the file you want the generated classes to go in (the default language is C#, but you can set it to use VB)

/uri - Obviously this is the URL of the data service you want to generate the client model for

The webdatagen tool has a lot more options than this, but I’m not going to go into them. You can type /help to get all that info. The important thing is that after running the above command, I now have a file called CellPhoneModel.cs that contains classes for Carrier, Manufacturer, and CellPhone (those of the entities in that service), as well as a nice WebDataContext. I include the file in my project and can now run the following code:

That’s definitely a nicer alternative to what we were using before. Our code is now more explicit, and we’ve abstracted how we’re accessing the CellPhone data using the newly generated property. If you wanted to take it a step further, you could either create a default constructor on your generated WebDataContext that infers the service URL from somewhere (hard-coded, config file, etc) or you could use an IoC container to inject the URL into the existing constructor. Either way, you can see how easy working with a data service can be.

If you’re curious how the CellPhones property works, here is the generated code:

As you can see it’s the same exact logic we were running manually before, except in this case it’s also being smart enough to “cache” the CellPhone data, which we weren’t doing before.

2) Lazy Loading

Now that we are working with strongly-typed objects, we no doubt want to be able to access both scalar and relational properties, otherwise what’s the point of having a client-side model?. As you may recall from article #3, when you request an entity from a data service, its relationships are not initially loaded, which is a good thing, because you could potentially have a lot of relationships on an entity, that could result in a ton of unneeded data being transfered.

When making a request through a URL in a browser, or using the WebDataContext’s CreateQuery method, you can use the $expand query parameter to eagerly load any relationships. This approach is great, but also has a couple draw backs:

1. You have to know at the time the query is made that you want to expand a relationship (or relationships).
2. If you are querying an entire entity set, and you want to expand a relationship property, you have to expand it for every single entity in the collection. If you only want the relationship expanded for certain entities, based on some type of criteria, there is no way to do that, and you’ll end up transferring a potentially large amount of unneeded data.

Imagine I want to make a query for all cell phones. Easy right? In fact, we already saw that example done very simply in the previous section. Let’s revisit it for the sake of repetition…

That will return to us a collection of CellPhone objects, but their Carrier and Manufacturer properties won’t be populated (thanks to lazy loading). We decide that we also want to receive the cell phone’s manufacturer, so we modify the above code:

Perfect! We now have our manufacturer data. We later figure out that we only need the manufacturer data for phones that are less than $500 and weights more than 3 ounces (that is a ridiculous need, but work with me here). For us to get manufacturer data using the CreateQuery method, it’s all or nothing in regards to retrieving relationship data. We know we need to get some manufacturer data, but we’re really hesitant to just blindly pull down all manufacturers when we only need a few. It’s clear that CreateQuery doesn’t offer the functionality we need.

Luckily, CreateQuery isn’t the only method the WebDataContext class has to offer. LoadProperty allows you to specify an entity object instance, and the name of one of its relationship properties that you want to populate. LoadProperty is great because it’s an “after the fact” method. It allows you to query the base data you know you need up front, and then later pull down just the extra stuff, as you need it. In our current example, we want to query all cell phones (without expanding any relationships), and then programatically determine which cell phones need their manufacturer data, and at that point retrieve it. The following code would work beautifully:

“Isn’t that going to make a bunch of individual calls to the data service to retrieve the manufacturer data? We may have saved bandwidth up front by not expanding manufacturers, but now we’re going to pay the processing price by making a lot of extra calls to the service.” That is very true. The above example wasn’t meant to say that LoadProperty is the best solution every time, nor is making a broad query with CreateQuery the best solution every time. The point is, you have two options for performing your lazy loading, each of which provides a different level of precision with your data. If only 3 out of 1,200 cell phones meet the above criteria, then there is a good chance that using LoadProperty will be more performance. But if 639 out of 1,200 cell phones meet the criteria, you might want to weight the advantages between bandwidth and server calls.

You may be thinking to yourself right now, “You know, I’ve gotten really comfortable with the way LINQ To SQL handles lazy loading of relationship properties. All I have to do is access the property, and behind the scenes it pulls down the data I need. I don’t have to make any extra method calls, it just works. LoadProperty seems like an extra step to me.” You’d be very right. LINQ To SQL (and the Entity Framework for that matter) does make the process of lazy loading a little more streamlined. But, have you ever taken a look at how LINQ To SQL accomplishes its lazy loading? It uses three helper classes (EntityRef<T>, EntitySet<T>, Link<T>) to represent the lazy loaded properties, and relies on code in the property’s getter/setter to work with the underlying helper class reference. It isn’t hard to understand/develop, but it’s added code and added complexity. With ADO.NET Data Services, you don’t need any helper classes, and you don’t need any extra code in your properties. Unfortunately you don’t get the extra behavior, but you do have less dependencies in your entity classes, and less bloat.

You could easily hand-code an ADO.NET Data Services client-model, because it comes very close to being true POCO. The same couldn’t really be said about a LINQ To SQL model, and certainly not for an EDM. I personally love the leanness of the data service client model, and I think a call to LoadProperty is a fair trade-off. In fact, you could even look at a call to LoadProperty as adding a little more explicitness to your code, that you wouldn’t have by just accessing the property and having the plumbing take care of the magic for you.

3) Identity Tracking

So why is identity tracking so important exactly? Well as you can imagine, every time a client requests data from a service, it requires time to return the response, and it requires bandwidth from the server to satisfy the request. In the case of Phillip’s Cell Phone Shack, he is in the business of making money, and having to serve the same cell phone data over and over again, when unnecessary, isn’t the greatest idea ever conceived. If the client was stateless (web application) then such tracking wouldn’t hold as much merit, but if the client was a desktop application, you can imagine that being able to hold a local cache of entity data would save a lot of needless round trips.

Once again, just like LINQ To SQL’s DataContext, and the Entity Framework’s ObjectContext, the WebDataContext class has identity tracking built into it. If you request a cell phone object, and then request the same cell phone again, it will be smart enough to give you the existing instance and not make the service call. This is not groundbreaking functionality by any means, but it’s a requirement for efficient applications, and it’s great that it’s a part of the ADO.NET Data Services feature set.

Above, I mentioned that data services work with pure POCO, and that statement is true as you saw. I also mentioned that there was an exception. In order for the WebDataContext to perform identity tracking, it has to know how to determine the identity of each entity type it manages. In order for it to do that, each entity type has to abide by the same rules as the entity type on the server. The only difference is that instead of applying the WebDataKey attribute to your class, you apply the Key attribute. So if I wanted to enable tracking for my Carrier entity, I could simply do:

If you hand-coded your entity classes, and don’t care about identity tracking, then you’re fine. If you hand-coded your entity classes and you do care about identity tracking, then you’ll need to make sure your entity’s identity property is apparent to the WebDataContext. If you used the webdatagen tool, it assumes you care about identity tracking and applies the Key attribute to the entity type’s identifying property, if it needs to. I say “if it needs to” because as you remember, if the identity property is “ID” or “[EntityType]ID” then that will automatically work, just like the server-side model types. In our case, none of my entity types follow either of those conventions, so I need to explicitly use the Key attribute. If your entity type’s identifying properties do in fact follow one of those conventions, then your data service client could work with pure POCO and leverage identity tracking as well. How sweet is that

So now that your entity classes are identity tracking enabled, there is still one more step before you can actually benefit from identity tracking. Because identity tracking is an additional feature that may not be needed by the client application, it is turned off by default. In order to enable it, you have to set the WebDataContext’s MergeOption property to something other that NoTracking (the default value). The following code would enable identity tracking for our context:

By setting the MergeOption to AppendOnly, you’re telling the WebDataContext to only append new entity instances that it doesn’t already have, effectively turning on identity tracking. The MergeOption enumeration has two other values: OverwriteChanges, and PreserveChanges. These values relate to the change tracking features of the WebDataContext.

4) Change Tracking

The unit of work pattern is great because it allows you to make a bunch of changes in a single session and then submit all those changes for persistence in one swoop. This is by no means new, in fact .NET developers have enjoyed it for years. LINQ To SQL’s DataContext, Entity Framework’s ObjectContext, and even the DataSet, all represent implementation’s of the unit of work pattern. Thankfully, the WebDataContext is no different. It allows us to make many different changes to its entity data (add/update/delete) and then perform a batch save. Best of all it will only persist the data and entities that have been modified, which means no unnecessary data being sent across the wire and no unnecessary logic being performed.

“Woah woah woah!”, you may be saying. “What is all this jibber jabber about changing entity data? Can we even do that with a data service? I thought this sucker was read only? How do you even perform updates to a service that is RESTfully exposed anyways?”. Those are all good points. The cool thing is, yes, you can make changes to a data service’s entity data from a client application. This topic deserves an article in itself, and as such will be tabled until part 7 of this series. Fear not faithful reader, you will wield these great powers very soon. Just be patient and let me finish explaining the entire client-service querying capabilities gamut.

So we’ve added the WebDataContext class to our arsenal of tools. We’ve learned how to leverage the webdatagen tool to build our lean client-side model. We’ve seen how to benefit from lazy loading, identity and change tracking, as well as get full object deserialization. One problem still exists though: a richer coding model for querying the data service seems in order.

If you’ve gotten used to using LINQ, no doubt you might see the above code samples as feeling a little primitive in comparison. Well, today is your lucky day, because there is a new LINQ flavor in town, and it goes by the name LINQ To Data Services. In the next article, we will see how we can leverage our knowledge of consuming data services and throw it away. LINQ is in baby! Everything else is out!