Temporary failure exception when using in multithreaded enviroment

Dec 15, 2010 at 4:43 PM
I deployed changed version of our product to production enviroment. All is working just fine, but under load i began to get these exceptions:
 Microsoft.ApplicationServer.Caching.DataCacheException: ErrorCode:SubStatus:There is a temporary failure. Please retry later. (There was a contention on the store.) at Microsoft.ApplicationServer.Caching.DataCache.ThrowException(ResponseBody respBody) at Microsoft.ApplicationServer.Caching.DataCache.InternalPut(String key, Object value, DataCacheItemVersion oldVersion, TimeSpan timeout, DataCacheTag[] tags, String region) at Microsoft.ApplicationServer.Caching.DataCache.Put(String key, Object value, TimeSpan timeout, String region) at CacheProvider.AppFabric.AppFabricCacheProvider.PutImpl(String key, Object value, TimeSpan timeout, String regionName) in C:\@Projects\ProviderCache\ProviderBasedCache\AppFabricCacheProvider\AppFabricCacheProvider.cs:line 410 

Investigated a bit and found few unswers, that this may be connected with simultaneous inserting with the same key. But everywhere in the code i firstly check if this value exists or not and i suppose that put method i use does not care if the value exists in cache or not.
This is a code i use:
CacheBroker.Put(BuildKey(key, keyArgs), data, _region);
Anoher question is how would you like that i test the performance of the product ? In our solution speed is not so important and cache is used to lower load of sql server.
Dec 16, 2010 at 4:37 PM

I will invesigate this a little more and let you know. However, my understanding is that if two threads or processes try to simultaneously add\update the same key at the very same time, this error is thrown by AppFabric Cache Cluster.  In that case, we will have to catch this exception for this error code and re-try.
If speed (e.g. no. of Gets\Puts per second) is not a concern, then, what is your required unit for measuring the performance (e.g. Reliability, High availability, High Volume, etc)?  If you give me some hint around this, I should be able to tell you as how to measure the same.
For testing no. of puts with dependency per second, there is a test PutTest_Performance() in the class AppFabricCacheProviderTest.  You can have a look at this test method.  Run it for 5 to 10 times and take an average.