WCF Data Services: Optimizing bandwidth usage and performance with updates
If you use WCF Data Services, you’ll probably know something about the OData protocol. For example, the OData protocol supports two kinds of updates:
- PUT requests: An entity is replaced with the received entity data. This means that all the entity data must be transferred over the wire.
- MERGE requests: Only the properties that are available in the request are updated on the server. Not all the properties of the entity have to be present in the request, only the ones that need to be updated.
The WCF Data Services client library uses MERGE by default, so you might think that the updates over the wire are optimized to only send the properties that have actually changed. Sadly, this isn’t the case and a lot of users aren’t aware of this. In this post I’m going to provide a solution which can really cut down on the bandwidth used by WCF Data Services when sending updates to the server.
The problem
Let’s take a look at a sample database:
It’s a small database but it illustrates the problem quite well. Now Let’s create an Entity Framework Data Model for this database:
Now create a WCF Data Service for it and enable access to the entity sets:
1: using System.Data.Services;
2: using System.Data.Services.Common;
3:
4: namespace ServiceHostSite
5: {
6: public class PetService : DataService<PetsEntities>
7: {
8:
9: public static void InitializeService(DataServiceConfiguration config)
10: {
11: config.SetEntitySetAccessRule("Pet", EntitySetRights.All);
12: config.SetEntitySetAccessRule("Owner", EntitySetRights.All);
13: config.UseVerboseErrors = true;
14: config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
15: }
16: }
17: }
This all leads to the following project:
Now we can add a client project, add a service reference to our service and test it:
1: using System;
2: using System.Linq;
3: using Beek.Van.Alex.DataServiceExtensions;
4: using ConsoleApplication1.Services;
5:
6: namespace ConsoleApplication1
7: {
8: class Program
9: {
10: static void Main(string[] args)
11: {
12: PetsEntities ents = new PetsEntities(new Uri("http://localhost.:51870/PetService.svc", UriKind.Absolute));
13: Pet soes = ents.Pet.Where((p) => p.PetId == 1).Single();
14:
15: soes.PetName = "A modified property";
16: ents.UpdateObject(soes);
17: ents.SaveChanges();
18:
19: Console.WriteLine("Finished!!!!!");
20: Console.ReadKey();
21: }
22: }
23: }
This should work and if we take a look at the database the data should be correctly updated. But, in a lot of scenario’s (ASP.NET, stateless services) you only get the updated data along with an id and it’s not acceptable to first send a request for the entity to retrieve it and then update it. Let’s change our code to simulate this:
1: using System;
2: using System.Linq;
3: using Beek.Van.Alex.DataServiceExtensions;
4: using ConsoleApplication1.Services;
5:
6: namespace ConsoleApplication1
7: {
8: class Program
9: {
10: static void Main(string[] args)
11: {
12: PetsEntities ents = new PetsEntities(new Uri("http://localhost.:51870/PetService.svc", UriKind.Absolute));
13: Pet soes = new Pet();
14:
15: soes.PetId = 1;
16: soes.PetName = "A modified property";
17:
18: ents.AttachTo("Pet", soes);
19: ents.UpdateObject(soes);
20: ents.SaveChanges();
21:
22: Console.WriteLine("Finished!!!!!");
23: Console.ReadKey();
24: }
25: }
26: }
27:
When running the above code you will get an exception:
When inspecting the SQL that was executed, the cause of the exception becomes clear:
1: exec sp_executesql N'update [dbo].[Pet]
2: set [PetName] = @0, [OwnerId] = @1
3: where ([PetId] = @2)
4: select [Version]
5: from [dbo].[Pet]
6: where @@ROWCOUNT > 0 and [PetId] = @2',N'@0 nvarchar(50),@1 int,@2 int',@0=N'A modified property',@1=0,@2=1
The Entity Framework tries to update all the columns, including the OwnerId column. Problem is that this property has the value 0, since I left it at it’s default value after constructing the new Pet object in the client. When we take a look with Fiddler, this is the data that has been transferred over the wire:
1: MERGE http://localhost.:51870/PetService.svc/Pet(1) HTTP/1.1
2: User-Agent: Microsoft ADO.NET Data Services
3: DataServiceVersion: 1.0;NetFx
4: MaxDataServiceVersion: 2.0;NetFx
5: Accept: application/atom+xml,application/xml
6: Accept-Charset: UTF-8
7: Content-Type: application/atom+xml
8: Host: localhost.:51870
9: Content-Length: 807
10: Expect: 100-continue
11: Connection: Keep-Alive
12:
13: <?xml version="1.0" encoding="utf-8" standalone="yes"?>
14: <entry >="http://schemas.microsoft.com/ado/2007/08/dataservices" >="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" >="http://www.w3.org/2005/Atom">
15: <category scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" term="PetsModel.Pet" />
16: <title />
17: <author>
18: <name />
19: </author>
20: <updated>2008-08-23T14:58:37.58344Z</updated>
21: <id>http://localhost.:51870/PetService.svc/Pet(1)</id>
22: <content type="application/xml">
23: <m:properties>
24: <d:OwnerId m:type="Edm.Int32">0</d:OwnerId>
25: <d:PetId m:type="Edm.Int32">1</d:PetId>
26: <d:PetName>A modified property</d:PetName>
27: <d:Version m:type="Edm.Binary" m:null="true" />
28: </m:properties>
29: </content>
30: </entry>
On line 1 you can see that this a MERGE request, but on lines 23-28 it becomes clear that the WCF Data Services client library always sends all the properties in the request, even when using a MERGE and updating only one property. This means that a MERGE essentially becomes the same as a PUT request, which of course, isn’t ideal, most certainly not when updating large objects. Note that this was also the case in the first example when I first retrieved the entity before updating, but it didn’t throw an exception because all the properties had their database values.
The solution
The culprit in this example is the UpdateObject() method in the DataServiceContext class. Without it, we can’t notify the context that an object has been modified. With it, the whole object becomes modified, it really is an all or nothing story. To make matters worse, the client library doesn’t have a lot of hooks which let us customize the serialization process or code generation process. I decided to model my solution similar to the solution that was used in the Entity Framework. In the Entity Framework the ObjectContext has an ObjectStateManager, which keeps track of the entities that have been modified. It does this with ObjectStateEntry objects. Each ObjectStateEntry keeps track of which properties have been modified for one entity. Here is an overview of the solution:
The role of the ObjectStateEntry is played by the UpdatedPropertiesEntry class, the role of the ObjectStateManager by the UpdatedPropertiesManager class. We’ll cover the other classes later, but let’s begin with the UpdatedPropertiesEntry class:
1: using System;
2: using System.Collections.Generic;
3: using System.Linq;
4: using System.Linq.Expressions;
5: using System.Reflection;
6:
7: namespace Beek.Van.Alex.DataServiceExtensions
8: {
9: internal class UpdatedPropertiesEntry
10: {
11: private object _target;
12: private Dictionary<PropertyInfo,Property> _properties;
13: internal object Target
14: {
15: get
16: {
17: return _target;
18: }
19: }
20:
21: internal IEnumerable<Property> Properties
22: {
23: get
24: {
25: return _properties.Values.ToArray();
26: }
27: }
28:
29:
30: internal UpdatedPropertiesEntry(object target)
31: {
32: if (target == null)
33: {
34: throw new ArgumentNullException("target");
35: }
36:
37: _target = target;
38: _properties = new Dictionary<PropertyInfo, Property>();
39: }
40:
41:
42:
43: internal void SetPropertyModified<T, TResult>(Expression<Func<T, TResult>> toAdd)
44: {
45: Property property = PropertyUtil.ConvertExpressionToProperty<T, TResult>(toAdd);
46:
47: AddProperty(property);
48:
49: }
50:
51: private void AddProperty(Property property)
52: {
53: _properties[property.PropertyInfo] = property;
54: }
55:
56: internal bool IsPropertyModified<T, TResult>(Expression<Func<T, TResult>> toCheck)
57: {
58:
59: PropertyInfoConversionResult result = PropertyUtil.ConvertExpressionToPropertyInfos(toCheck);
60:
61: return IsPropertyModified(result);
62: }
63:
64: private bool IsPropertyModified(PropertyInfoConversionResult result)
65: {
66: bool toReturn;
67: Property property;
68: if (result.IsComplexProperty)
69: {
70: toReturn = IsPropertyOfComplexPropertyModified(result.ComplexPropertyProperty, result.SimplePropertyProperty);
71: }
72: else
73: {
74: toReturn = _properties.TryGetValue(result.SimplePropertyProperty, out property);
75: }
76: return toReturn;
77: }
78:
79: private bool IsPropertyOfComplexPropertyModified(PropertyInfo complexProperty,PropertyInfo simplePropertyInfo)
80: {
81: bool toReturn;
82: Property property;
83:
84: toReturn = _properties.TryGetValue(complexProperty, out property);
85: if (toReturn)
86: {
87: toReturn = ((ComplexProperty)property).IsPropertyModified(simplePropertyInfo);
88: }
89: return toReturn;
90: }
91:
92: internal bool IsPropertyOfComplexPropertyModified(string complexPropertyName, string simplePropertyName)
93: {
94: PropertyInfo simplePropertyInfo;
95: PropertyInfo complexPropertyInfo;
96:
97: complexPropertyInfo = PropertyUtil.GetPropertyInfoByName(_target.GetType(), complexPropertyName);
98: simplePropertyInfo = PropertyUtil.GetPropertyInfoByName(complexPropertyInfo.PropertyType, simplePropertyName);
99:
100: return IsPropertyOfComplexPropertyModified(complexPropertyInfo, simplePropertyInfo);
101: }
102:
103:
104:
105:
106: internal void ClearPropertyModified<T, TResult>(Expression<Func<T, TResult>> toClear)
107: {
108: PropertyInfoConversionResult result = PropertyUtil.ConvertExpressionToPropertyInfos(toClear);
109: ClearPropertyModified(result);
110: }
111:
112: private void ClearPropertyModified(PropertyInfoConversionResult result)
113: {
114: Property property;
115:
116: if (result.IsComplexProperty)
117: {
118: if (_properties.TryGetValue(result.ComplexPropertyProperty, out property))
119: {
120: ComplexProperty complexProperty = (ComplexProperty)property;
121: complexProperty.UnModifyProperty(result.SimplePropertyProperty);
122: if (!complexProperty.HasModifiedProperties())
123: {
124: _properties.Remove(result.ComplexPropertyProperty);
125: }
126: }
127: }
128: else
129: {
130: _properties.Remove(result.SimplePropertyProperty);
131: }
132: }
133:
134:
135: internal bool IsPropertyModified(string propertyString)
136: {
137: PropertyInfoConversionResult result = PropertyUtil.ConvertStringToPropertyInfos(_target,propertyString);
138: return IsPropertyModified(result);
139: }
140:
141: internal void ClearPropertyModified(string toClear)
142: {
143: PropertyInfoConversionResult result = PropertyUtil.ConvertStringToPropertyInfos(_target,toClear);
144: ClearPropertyModified(result);
145:
146: }
147:
148: internal void SetPropertyModified(string property)
149: {
150: Property propertyToAdd = PropertyUtil.ConvertStringToProperty(_target, property);
151: AddProperty(propertyToAdd);
152: }
153:
154: internal bool HasModifiedProperties()
155: {
156: return _properties.Count != 0;
157: }
158:
159: internal bool IsModifiedPropertyComplex(string propertyName)
160: {
161: Property property;
162: PropertyInfo info = PropertyUtil.GetPropertyInfoByName(_target.GetType(), propertyName);
163: bool toReturn;
164:
165: toReturn = _properties.TryGetValue(info, out property);
166: if (toReturn)
167: {
168: toReturn = property.PropertyType == PropertyType.Complex;
169: }
170: else
171: {
172: throw new ArgumentException("The specified property has not been flagged as a simple property that has been modified or as a complex property whose properties have been modified!");
173: }
174: return toReturn;
175: }
176:
177: }
178: }
The responsibility of this class is to keep track for an entity which properties have been modified. The entity is supplied in the constructor of this class. The class has a lot of methods but most interesting is the SetPropertyModified method on line 43. This method accepts a generic Expression of Func. I decided to use this so that the user of my API can supply a lambda expression to indicate which property has been modified. Modified properties are instances of two classes, SimpleProperty or ComplexProperty (comparable to scalar properties and complex properties in the Entity Framework), both inherit from the abstract Property class. A SimplePropery is not much more than a wrapper around a PropertyInfo object, a ComplexProperty is a wrapper around a collection of PropertyInfo objects. Each property in a ComplexProperty can be modified independently.
The heavy lifting is done in the ConvertExpressionToProperty method of the PropertyUtil class. This class defines a couple of convenience methods that can convert a lambda expression or a property path string to the correct Property objects. Thus, a SimpleProperty if the Expression doesn’t contain a ‘.’ and ComplexProperty if the lambda does.
So let’s take a look at the ConvertExpressionToProperty () method on line 9:
1: using System;
2: using System.Linq.Expressions;
3: using System.Reflection;
4:
5: namespace Beek.Van.Alex.DataServiceExtensions
6: {
7: internal static class PropertyUtil
8: {
9: internal static Property ConvertExpressionToProperty<T, TResult>(Expression<Func<T, TResult>> toConvert)
10: {
11: Property toReturn;
12: PropertyInfoConversionResult result = ConvertExpressionToPropertyInfos(toConvert);
13:
14: if (result.IsComplexProperty)
15: {
16: toReturn = ConvertPropertyInfosToComplexProperty(result.ComplexPropertyProperty, result.SimplePropertyProperty);
17:
18: }
19: else
20: {
21: toReturn = ConvertPropertyInfoToSimpleProperty(result.SimplePropertyProperty);
22: }
23:
24: return toReturn;
25:
26: }
27:
28: internal static Property ConvertStringToProperty(object target, string toConvert)
29: {
30: Property toReturn;
31: PropertyInfoConversionResult result = ConvertStringToPropertyInfos(target, toConvert);
32:
33: if (result.IsComplexProperty)
34: {
35: toReturn = ConvertPropertyInfosToComplexProperty(result.ComplexPropertyProperty, result.SimplePropertyProperty);
36:
37: }
38: else
39: {
40: toReturn = ConvertPropertyInfoToSimpleProperty(result.SimplePropertyProperty);
41: }
42:
43: return toReturn;
44:
45: }
46:
47: private static SimpleProperty ConvertPropertyInfoToSimpleProperty(PropertyInfo property)
48: {
49: ValidateSimplePropertyInfo(property);
50: return new SimpleProperty(property);
51: }
52:
53: private static void ValidateSimplePropertyInfo(PropertyInfo property)
54: {
55: if (property.PropertyType != typeof(String) && property.PropertyType != typeof(byte[]) &&
56: !property.PropertyType.IsValueType)
57: {
58: throw new ArgumentException("Simple property must be a value type, String or Byte[]");
59: }
60: }
61:
62: private static ComplexProperty ConvertPropertyInfosToComplexProperty(PropertyInfo complexProperty, PropertyInfo simpleProperty)
63: {
64: ValidateSimplePropertyInfo(simpleProperty);
65: return new ComplexProperty(complexProperty, simpleProperty);
66: }
67:
68: internal static PropertyInfoConversionResult ConvertExpressionToPropertyInfos<T, TResult>(Expression<Func<T, TResult>> toConvert)
69: {
70: MemberExpression body = toConvert.Body as MemberExpression;
71: PropertyInfo simpleProperty;
72: PropertyInfo complexProperty = null;
73:
74: if (toConvert == null)
75: {
76: throw new ArgumentNullException("toAdd");
77: }
78:
79: if (body == null)
80: {
81: throw new ArgumentException("Must access a property in expression body!");
82: }
83:
84: simpleProperty = body.Member as PropertyInfo;
85: if (simpleProperty == null)
86: {
87: throw new ArgumentException("Must access a property in expression body!");
88:
89: }
90: if (body.Expression.NodeType != ExpressionType.Parameter)
91: {
92: //Must be a complex property then....
93: MemberExpression complexPropertyExpression = body.Expression as MemberExpression;
94: if (complexPropertyExpression == null)
95: {
96: throw new ArgumentException("Expression must access a property!!");
97: }
98: if (complexPropertyExpression.Expression.NodeType != ExpressionType.Parameter)
99: {
100: throw new ArgumentException("ComplexProperty expressions can only have a depth of one!");
101: }
102: complexProperty = complexPropertyExpression.Member as PropertyInfo;
103: if (complexProperty == null)
104: {
105: throw new ArgumentException("Expression must access a property!!");
106: }
107: }
108:
109:
110: return new PropertyInfoConversionResult(simpleProperty, complexProperty);
111: }
112:
113: internal static PropertyInfoConversionResult ConvertStringToPropertyInfos(object target,string propertyString)
114: {
115: Type entityType = target.GetType();
116: PropertyInfo simpleProperty;
117: PropertyInfo complexProperty = null;
118:
119: string[] properties;
120:
121: if (propertyString == null)
122: {
123: throw new ArgumentNullException("property");
124: }
125:
126: if (propertyString.Contains("."))
127: {
128: //Complex property
129: properties = propertyString.Split(new char[] { '.' }, StringSplitOptions.RemoveEmptyEntries);
130: if (properties.Length > 2)
131: {
132: throw new ArgumentException("Complex properties can only be one deep!");
133: }
134:
135: complexProperty = GetPropertyInfoByName(target.GetType(),properties[0]);
136: simpleProperty = GetPropertyInfoByName(complexProperty.PropertyType, properties[1]);
137:
138: } else
139: {
140: simpleProperty = GetPropertyInfoByName(target.GetType(), propertyString);
141: }
142:
143: return new PropertyInfoConversionResult(simpleProperty,complexProperty);
144: }
145:
146: internal static PropertyInfo GetPropertyInfoByName(Type targetType, string propertyName)
147: {
148: PropertyInfo toReturn;
149: toReturn = targetType.GetProperty(propertyName);
150: if (toReturn == null)
151: {
152: throw new ArgumentException(targetType + " Does not have a property with name: " + propertyName);
153: }
154: return toReturn;
155: }
156: }
157: }
On line 12 it turns the supplied lambda expression in one PropertyInfo object if it’s a simple expression, it turns it into two PropertyInfo objects if the expression is an expression that points to a property of a complex property of an entity. The convert methods on line 16 and 21 are quite easy and are defined on lines 47 and 62. The real work is done in the ConvertExpressionToPropertyInfos() method. This method is defined on line 68.
It first does a couple of checks. The expression body is converted to a MemberExpression, if it’s not a MemberExpression an exception is thrown on line 79. On line 84 it retrieves the MemberInfo object of the MemberExpression and cast’s it to a PropertyInfo object. If it’s not a property, again an exception is thrown. Remember that an expression can point to a property of a complex property, for example: (Person p)=> p.Adddress.Zipcode. The property gotten on 84 is a PropertyInfo object that describes the ZipCode property. On line 90 we determine whether the the expression before the Zipcode (Address.) accesses the lambda parameter. In this case it doesn’t, since ZipCode is a property of Address, not Person. When it doesn’t, this must mean that a complex property is accessed and we can extract another PropertyInfo object, in this case, an object which describes the Address property. this is done on line 102. Finally the two PropertyInfo objects are wrapped in an object and returned. Note that I’ve defined a similar method on line 113 which can accept a property path string to indicate a modified property.
So we have an UpdatedPropertiesEntry object, which holds Property (SimpleProperty or ComplexProperty) objects per entity. Users can indicate to an UpdatedPropertiesEntry object whether a property has changed with a strongly typed lambda expression or a property path string. Next to that, the UpdatedPropertiesEntry object has methods to “UnModify” a property and to check whether a property was modified. This can always be a simple property of an entity, or a simple property of a complex property of an entity. All the methods have overloads to support both a strongly typed lambda expression as well as a property path string.
In order for this all to work, there must be an object which keeps track of all the UpdatedPropertyEntry objects for a DataServiceContext per entity. This is the responsibility of the UpdatedPropertiesManager class (I’ve included the source of this class as a link, since my blog engine can’t handle it within this post, so open it in a new tab :)).
This class is essentially a wrapper around a dictionary. This dictionary associates each entity with an UpdatedPropertiesEntry object. Furthermore, this class has a couple convenience methods which wraps methods of the UpdatedPropertiesEntry class:
- SetPropertyModified
- IsPropertyModified
- IsEntityModified
- ClearEntities
- IsModifiedPropertyComplex
Most of these methods are nothing more than retrieving the UpdatedPropertiesEntry object for an entity and delegating to this object. The SetPropertyModified method on line 104 for example, checks whether an entry for the entity exists, if not creates it and then calls the SetPropertyModified method on this entry. The methods in this class also support lambda expressions and property path strings.
Gluing everything together
Okay, We’ve got an UpdatedPropertiesEntry, which can hold which properties have been modified for an entity and we’ve got an UpdatedPropertiesManager which keeps track of all the entities and their corresponding UpdatedPropertiesEntry objects. Now we need to make sure that the DataServiceContext uses this UpdatedPropertiesManager. I wanted a solution that played nicely with the code that was generated by Visual Studio when you add a service reference to a WCF Data Service. The only way you can extend the generated code is by creating another partial DataServiceContext class. I decided that if you want to use my API, the DataServiceContext must implement an interface:
1:
2: namespace Beek.Van.Alex.DataServiceExtensions
3: {
4: public interface IDataServiceWithPropertiesManager
5: {
6: UpdatedPropertiesManager PropertiesManager
7: {
8: get;
9: }
10: }
11: }
You can let your client side generated DataServiceContext class implement this interface as follows:
1: using Beek.Van.Alex.DataServiceExtensions;
2:
3: namespace ConsoleApplication1.Services
4: {
5: partial class PetsEntities : IDataServiceWithPropertiesManager
6: {
7: private UpdatedPropertiesManager _propertiesManager;
8:
9: public UpdatedPropertiesManager PropertiesManager
10: {
11: get { return _propertiesManager ?? (_propertiesManager = new UpdatedPropertiesManager()); }
12: }
13: }
14: }
Now if there was only a way to automatically add functionality to a class as soon as it implements a certain interface……Oh but there is! They are of course extension methods and are defined in the DataServiceExtensions class:
1: using System;
2: using System.Data.Services.Client;
3: using System.Diagnostics;
4: using System.Linq;
5: using System.Linq.Expressions;
6: using System.Xml.Linq;
7:
8: namespace Beek.Van.Alex.DataServiceExtensions
9: {
10: public static class DataServiceExtensions
11: {
12:
13: public static bool IsEntityModified<T>(this T context, object entity)
14: where T : DataServiceContext, IDataServiceWithPropertiesManager
15: {
16: ValidateContext(context);
17: return context.PropertiesManager.IsEntityModified(entity);
18: }
19:
20: public static bool IsPropertyModified<T, TEntity, TResult>(this T context, TEntity entity, Expression<Func<TEntity, TResult>> modifiedPropertyExpression)
21: where T : DataServiceContext, IDataServiceWithPropertiesManager
22: {
23: ValidateContext(context);
24: return context.PropertiesManager.IsPropertyModified(entity, modifiedPropertyExpression);
25: }
26:
27: public static bool IsPropertyModified<T>(this T context, object entity,string modifiedProperty)
28: where T : DataServiceContext, IDataServiceWithPropertiesManager
29: {
30: ValidateContext(context);
31: return context.PropertiesManager.IsPropertyModified(entity, modifiedProperty);
32: }
33:
34: public static void ClearModification<T>(this T context, object entity)
35: where T : DataServiceContext, IDataServiceWithPropertiesManager
36: {
37: ValidateContext(context);
38: EntityDescriptor descriptor = context.GetEntityDescriptor(entity);
39: string eTag;
40: string entitySet;
41: string[] uriSegments;
42: string segment;
43:
44: if (descriptor.State != EntityStates.Modified)
45: {
46: throw new ArgumentException("Can only clear modification for modified entities!");
47: }
48:
49: //Only way to "UnModify" an object is to attach and reattach it, with optimistic concurrency the etag is needed
50: eTag = descriptor.ETag;
51:
52: //Getting entitysetname out of edit uri, needed for attach
53: uriSegments = descriptor.EditLink.Segments;
54: segment = uriSegments[uriSegments.Length - 1];
55: entitySet = segment.Substring(0, segment.IndexOf("("));
56:
57: context.Detach(entity);
58: context.AttachTo(entitySet, entity, eTag);
59: Debug.Assert(context.GetEntityDescriptor(entity).State == EntityStates.Unchanged, "Must be in the unchanged state after attaching!");
60: context.PropertiesManager.ClearEntry(entity);
61: }
62:
63: public static void SetPropertyModified<T, TEntity, TResult>(this T context, TEntity entity, Expression<Func<TEntity, TResult>> modifiedPropertyExpression)
64: where T : DataServiceContext, IDataServiceWithPropertiesManager
65: {
66: ValidateContext(context);
67: context.UpdateObject(entity);
68: context.PropertiesManager.SetPropertyModified(entity, modifiedPropertyExpression);
69: }
70:
71: public static void UnModifyProperty<T, TEntity, TResult>(this T context, TEntity entity, Expression<Func<TEntity, TResult>> modifiedPropertyExpression)
72: where T : DataServiceContext, IDataServiceWithPropertiesManager
73: {
74: ValidateContext(context);
75: context.PropertiesManager.UnModifyProperty(entity, modifiedPropertyExpression);
76: if (!context.PropertiesManager.IsEntityModified(entity))
77: {
78: ClearModification(context, entity);
79: }
80: }
81: public static void UnModifyProperty<T>(this T context, object entity, string propertyPath)
82: where T : DataServiceContext, IDataServiceWithPropertiesManager
83: {
84: ValidateContext(context);
85: context.PropertiesManager.UnModifyProperty(entity, propertyPath);
86: if (!context.PropertiesManager.IsEntityModified(entity))
87: {
88: ClearModification(context, entity);
89: }
90: }
91:
92: public static void SetPropertyModified<T>(this T context, object entity, string modifiedProperty)
93: where T : DataServiceContext, IDataServiceWithPropertiesManager
94: {
95: ValidateContext(context);
96: context.UpdateObject(entity);
97: context.PropertiesManager.SetPropertyModified(entity, modifiedProperty);
98: }
99:
100: public static void OptimizedSaveChanges<T>(this T context, SaveChangesOptions option )
101: where T : DataServiceContext, IDataServiceWithPropertiesManager
102: {
103: if(option == SaveChangesOptions.ReplaceOnUpdate)
104: {
105: throw new ArgumentException("Can only optimize for MERGE requests!");
106: }
107: ValidateContext(context);
108: DataServiceContext localContext = (DataServiceContext)context;
109: context.WritingEntity += OptimizeMessageBeforeSend;
110:
111: try
112: {
113: localContext.SaveChanges(option);
114: //When everything has succeeded clear the modified entities....
115: ((IDataServiceWithPropertiesManager)localContext).PropertiesManager.ClearEntities();
116: }
117: catch (DataServiceRequestException ex)
118: {
119: if (option != SaveChangesOptions.Batch)
120: {
121: //When in batch everything fails, so no clean up needed, otherwise:
122: DataServiceResponse response = ex.Response;
123: foreach (ChangeOperationResponse item in response)
124: {
125: if (item.Error == null)
126: {
127: //Changes were persisted so remove modified properties
128: EntityDescriptor descriptor = (EntityDescriptor)item.Descriptor;
129: context.PropertiesManager.ClearEntry(descriptor.Entity);
130: }
131: }
132: }
133: throw ex;
134: }
135: finally
136: {
137: context.WritingEntity -= OptimizeMessageBeforeSend;
138: }
139: }
140:
141: private static void ValidateContext(IDataServiceWithPropertiesManager context)
142: {
143: if (context == null)
144: {
145: throw new ArgumentNullException("The supplied DataServiceContext can't be null!");
146: }
147:
148: if (context.PropertiesManager == null)
149: {
150: throw new ArgumentException("You can only call this method after the PropertiesManager of the DataServiceContext has been initialized!");
151: }
152: }
153:
154: private static void OptimizeMessageBeforeSend(object sender, ReadingWritingEntityEventArgs e)
155: {
156:
157: IDataServiceWithPropertiesManager contextProvider = (IDataServiceWithPropertiesManager)sender;
158: DataServiceContext context = (DataServiceContext)contextProvider;
159:
160: if (context.GetEntityDescriptor(e.Entity).State != EntityStates.Modified || !contextProvider.PropertiesManager.IsEntityModified(e.Entity))
161: {
162: // Only optimize when performing updates.....
163: return;
164: }
165:
166: XNamespace nameSpace = "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata";
167:
168: XElement properties = e.Data.Descendants(nameSpace + "properties").Single();
169: var propertyElements = properties.Elements().ToArray();
170:
171: foreach (XElement property in propertyElements)
172: {
173: if (!contextProvider.PropertiesManager.IsPropertyModified(e.Entity, property.Name.LocalName))
174: {
175: property.Remove();
176: }
177: }
178: }
179: }
180: }
Most of these methods are wrappers around the methods in the UpdatedPropertiesManager class, take a look at the SetPropertyModified() method in line 63 for example. What you should notice is that it has three generic parameters:
- T: This is needed because my methods only apply to DataServiceContext classes which implement my IDataServiceWithPropertiesManager interface. You can only specify these two constraints with generics.
- TEntity: This is the type of the entity that must be supplied and is inferred from the second argument (the first when you call the method using Extension method syntax).
- TResult: The return type of the supplied lambda expression, which must be generic since I don’t know which type of property it’s going to point at.
Most methods in this class look a lot like the SetPropertyModified() method, so I’m not going to cover them all. There are two interesting methods left. The first is the ClearModifcation() method on line 34. This method is used to reset the state of an entity to UnChanged and is called automatically after all the modified properties have been “UnModified”. You should notice three things:
- You can only set the state for an entity to UnChanged by detaching and attaching it.
- When optimistic concurrency is used you need the ETag when attaching. You can get this from the EntityDescriptor.
- You need to know the entity set name when attaching and this isn’t always the same as the class name. I extract it from the link which is used when an entity needs to be updated, which you can also get from the EntityDescriptor of an entity.
Second important method is OptimizedSaveChanges() on line 100. This is the Extension method a user of my API should call when they want to send all the changes to the database, without sending all the properties of an entity, but only the modified ones. This method attaches an eventhandler to the WritingEntity event of the DataServiceContext class. Then it saves all changes. After the changes have been saved it clears out all it’s entities and their entries since all the entities go back to the UnChanged state. If an exception occurs, dependent on the SaveChangesOptions value, it extracts information from the response about which entities were saved and which weren’t. The ones that were, have their modified properties cleared. Finally it removes the eventhandler for the WritingEntity event. As you’ve probably guessed, all the heavy lifting is done in the eventhandler for the WritingEntity event.
The WritingEntity eventhandler starts on line 154. The interesting parts:
- Line 160: Optimizations will only occur when an entity is being updated. The WritingEntity event also fires for inserts, so a check is needed.
- Line 166: This is the namespace declaration Microsoft uses in OData for metadata. If you scroll back the part where I showed you what was transferred over the wire with Fiddler, you can find this declaration on line 14 with the “m” prefix.
- Line 168: I extract the “properties” element. You can find this element in the “Fiddlered” request on line 23.
- Lines 171-177: I iterate over all the child elements of the properties element previously extracted. These are all the properties of an entity and not only the modified ones. I check with the help of my UpdatedPropertiesManager whether a property has been modified, if not, I remove it from the XML.
I don’t feel like this is the nicest solution, but I think it’s the only solution. The problem is that you can only change the XML that will be sent to the server after it has been created by the DataServiceContext class. I’d much rather create the correct XML in the first place, but this not possible at the moment with the WCF Data Services client library. Also note that if a property of a complex property is modified, all the properties of the complex property are sent to the server. This is done on purpose, since if you don’t send all the properties of a complex property to the server, WCF Data Services updates them with their default .Net values. This seems to be a bug in WCF Data Services and as soon as this is fixed I’ll update the library, since it already has full support for complex properties.
Using the DataServiceExtensions
Using my extensions is quite easy. First make sure that the generated DataServiceContext class implements my IDataServiceWithPropertiesManager interface as shown in my before last code example. After this you can use my API as follows:
1: using System;
2: using System.Linq;
3: using Beek.Van.Alex.DataServiceExtensions;
4: using ConsoleApplication1.Services;
5:
6: namespace ConsoleApplication1
7: {
8: class Program
9: {
10: static void Main(string[] args)
11: {
12: PetsEntities ents = new PetsEntities(new Uri("http://localhost.:51870/PetService.svc", UriKind.Absolute));
13: Pet soes = new Pet();
14: Pet milo = ents.Pet.Where((p) => p.PetId == 2).Single();
15: Pet toAdd = new Pet();
16:
17: soes.PetId = 1;
18: soes.PetName = "A modified property in soes";
19:
20: milo.PetName = "A modified property in Milo";
21: milo.OwnerId = 1;
22:
23: toAdd.PetName = "Lewy";
24: toAdd.OwnerId = 2;
25:
26: ents.AttachTo("Pet", soes);
27: ents.SetPropertyModified(soes, (p) => p.PetName);
28: ents.SetPropertyModified(milo, (p) => p.PetName);
29: ents.SetPropertyModified(milo, (p) => p.OwnerId);
30: ents.AddToPet(toAdd);
31:
32: ents.OptimizedSaveChanges(SaveChangesOptions.Batch);
33:
34: Console.WriteLine("Finished!!!!!");
35: Console.ReadKey();
36: }
37: }
38: }
39:
Note that in the example above I’ve included a Pet to insert, just for testing. What has changed when using my API:
- Line 3: You need this using statement to bring my extension methods in scope
- Line 27-29: You don’t call UpdateObject() anymore, but SetPropertyModified() instead, indicating which property has been modified with a strongly typed lambda expression. You can see that one property was modified for the pet “soes” and two for “milo”;
- Line 32: You don’t call SaveChanges() anymore, but my OptimizedSaveChanges() extension method.
Easy isn’t it? here is the “Fiddlered” request:
1: POST http://localhost.:51870/PetService.svc/$batch HTTP/1.1
2: User-Agent: Microsoft ADO.NET Data Services
3: DataServiceVersion: 1.0;NetFx
4: MaxDataServiceVersion: 2.0;NetFx
5: Accept: application/atom+xml,application/xml
6: Accept-Charset: UTF-8
7: Content-Type: multipart/mixed; boundary=batch_fab02648-1fe8-4bb8-ba40-11b03cc33a58
8: Host: localhost.:51870
9: Content-Length: 3110
10: Expect: 100-continue
11:
12: --batch_fab02648-1fe8-4bb8-ba40-11b03cc33a58
13: Content-Type: multipart/mixed; boundary=changeset_89dc03e8-aae4-4977-8aa3-fd9516150c13
14:
15: --changeset_89dc03e8-aae4-4977-8aa3-fd9516150c13
16: Content-Type: application/http
17: Content-Transfer-Encoding: binary
18:
19: MERGE http://localhost.:51870/PetService.svc/Pet(1) HTTP/1.1
20: Content-ID: 3
21: Content-Type: application/atom+xml;type=entry
22: Content-Length: 649
23:
24: <?xml version="1.0" encoding="utf-8"?>
25: <entry >="http://schemas.microsoft.com/ado/2007/08/dataservices" >="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" >="http://www.w3.org/2005/Atom">
26: <category scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" term="PetsModel.Pet" />
27: <title />
28: <author>
29: <name />
30: </author>
31: <updated>2008-08-23T19:17:26.4787264Z</updated>
32: <id>http://localhost.:51870/PetService.svc/Pet(1)</id>
33: <content type="application/xml">
34: <m:properties>
35: <d:PetName>A modified property in soes</d:PetName>
36: </m:properties>
37: </content>
38: </entry>
39: --changeset_89dc03e8-aae4-4977-8aa3-fd9516150c13
40: Content-Type: application/http
41: Content-Transfer-Encoding: binary
42:
43: MERGE http://localhost.:51870/PetService.svc/Pet(2) HTTP/1.1
44: Content-ID: 4
45: Content-Type: application/atom+xml;type=entry
46: Content-Length: 700
47:
48: <?xml version="1.0" encoding="utf-8"?>
49: <entry >="http://schemas.microsoft.com/ado/2007/08/dataservices" >="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" >="http://www.w3.org/2005/Atom">
50: <category scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" term="PetsModel.Pet" />
51: <title />
52: <author>
53: <name />
54: </author>
55: <updated>2008-08-23T19:17:26.4847456Z</updated>
56: <id>http://localhost.:51870/PetService.svc/Pet(2)</id>
57: <content type="application/xml">
58: <m:properties>
59: <d:OwnerId m:type="Edm.Int32">1</d:OwnerId>
60: <d:PetName>A modified property in Milo</d:PetName>
61: </m:properties>
62: </content>
63: </entry>
64: --changeset_89dc03e8-aae4-4977-8aa3-fd9516150c13
65: Content-Type: application/http
66: Content-Transfer-Encoding: binary
67:
68: POST http://localhost.:51870/PetService.svc/Pet HTTP/1.1
69: Content-ID: 5
70: Content-Type: application/atom+xml;type=entry
71: Content-Length: 731
72:
73: <?xml version="1.0" encoding="utf-8"?>
74: <entry >="http://schemas.microsoft.com/ado/2007/08/dataservices" >="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" >="http://www.w3.org/2005/Atom">
75: <category scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" term="PetsModel.Pet" />
76: <title />
77: <author>
78: <name />
79: </author>
80: <updated>2008-08-23T19:17:26.4857488Z</updated>
81: <id />
82: <content type="application/xml">
83: <m:properties>
84: <d:OwnerId m:type="Edm.Int32">2</d:OwnerId>
85: <d:PetId m:type="Edm.Int32">0</d:PetId>
86: <d:PetName>Lewy</d:PetName>
87: <d:Version m:type="Edm.Binary" m:null="true" />
88: </m:properties>
89: </content>
90: </entry>
91: --changeset_89dc03e8-aae4-4977-8aa3-fd9516150c13--
92: --batch_fab02648-1fe8-4bb8-ba40-11b03cc33a58—
This is an OData batch request as you can see on lines 12 and 13. On line 19 you see the first MERGE request for the Pet with id 1. This is Pet “soes” and she had one property changed, namely “PetName”. Now scroll to line 35. You can see that only the “PetName” property is present in the XML. On line 43 you can see the second MERGE request for Pet with id 2. This is “Milo” and he had two properties changed: “PetName” and “OwnerId”. Now scroll to lines 59 and 60. You can see that only these changed properties are present in the XML. On line 68 is the last request. This is a POST request and indicates an insert. All the properties should be present for an insert and if you take a look at lines 84-87, you can see that this is the case. This optimization can really save you a lot of bandwidth usage. Just try it with the Product table from the AdventureWorks database for example.
Concluding
Well I think the library is pretty complete for a first release but I still have a couple of good ideas that would make great WCF Data Service Extensions as well:
- An ETag generator at the client.
- Extend above example with Automatic Change Tracking when objects are attached or materialized.
- A Silverlight version of above project.
- Strongly typed Expand() method.
- Methods on the DataServiceContext to call [WebInvoke] service operations.
As always, the code is posted without warranties. I’ve tested my code pretty good, with all kinds of different databases and scenario’s, but I’m sure that I’ve still missed some scenarios and there may still be some bugs present.
You can find my WCF Data Services Extensions here.
And before I forget…….
Happy New Year!!!!!