Recipe3.29.Creating an Object Cache | Secure Programming Cookbook for C and C++: Recipes for Cryptography, Authentication, Input Validation & More

Recipe 3.29. Creating an Object Cache

problem

Your application creates many objects that are expensive to create and/or have a large memory footprintfor instance, objects that are populated with data from a database or a web service upon their creation. These objects are used throughout a large portion of the application's lifetime. You need a way to not only enhance the performance of these objectsand as a result, your applicationbut also to use memory more efficiently.

Solution

Create an object cache to keep these objects in memory as long as possible, without tying up valuable heap space and possibly resources. Since cached objects may be reused at a later time, you also forego the process of having to create similar objects many times.

You can reuse the ASP.NET cache that is located in the System.Web.Caching namespace, or you can build your own lightweight caching mechanism. The See Also section at the end of this recipe provides several Microsoft resources that show you how to use the ASP.NET cache to cache your own objects. However, the ASP.NET cache is very complex and may have a nontrivial overhead associated with it, so using a lightweight caching mechanism like the one shown here is a viable alternative.

The ObjCache<T,U> class shown in Example 3-19 represents a type that allows the caching of any type of object defined by parameter type U with a key defined by parameter type T.

Example 3-19. Implementing a generic object cache

 using System; using System.Collections; using System.Collections.Generic; public class ObjCache<T, U>     where U: new() {     // Constructors     public ObjCache()     {         cache = new Dictionary<T, WeakReference>();     }     public ObjCache(int initialCapacity)     {         cache = new Dictionary<T, WeakReference>(initialCapacity);     }     // Fields     private Dictionary<T, WeakReference> cache = null;     // Methods     public U this[T key]     {         get         {              if (!cache.ContainsKey(key) || !IsObjAlive(ref key))              {                  this[key] = new U();              }              return ((U)((WeakReference)cache[key]).Target);         }         set         {              WeakReference WR = new WeakReference(value, false);         }     }     public bool IsObjAlive(ref T key)     {         if (cache.ContainsKey(key))         {            return (((WeakReference)cache[key]).IsAlive);         }         else         {            return (false);         }     }     public int AliveObjsInCache()     {         int count = 0;         foreach (KeyValuePair<T, WeakReference> item in cache)         {             if (((WeakReference)item.Value).IsAlive)             {                 count++;             }         }         return (count);     }     public bool DoesKeyExist(T key)     {         return (cache.ContainsKey(key));     }     public bool DoesObjExist(WeakReference obj)     {         return (cache.ContainsValue(obj));     }     public int TotalCacheSlots()     {         return (cache.Count);     } }

The SomeComplexObj class can be replaced with any type of class you choose. For this recipe, you will use this class, but for your code, you can change it to whatever class or structure type you need.

The SomeComplexObj is defined here (realistically, this would be a much more complex object to create and use; however, for the sake of brevity, this class is written as simply as possible):

 public class SomeComplexObj {     public SomeComplexObj( ) {}     private int idcode = -1;     public int IDCode     {         set{idcode = value;}         get{return (idcode);}     } }

ObjCache<T,U>, the caching object used in this recipe, makes use of a Dictionary<T,WeakReference> object to hold all cached objects. This Dictionary<T,WeakReference> allows for fast lookup when retrieving objects and generally for fast insertion and removal times. The Dictionary<T,WeakReference> object used by this class is defined as a private field and is initialized through its overloaded constructors.

Developers using this class will mainly be adding and retrieving objects from this object. The indexer implements both the adding and retrieval mechanisms for this class. This method returns a cached object if its key exists in the Dictionary<T,WeakReference> and the WeakReference object is considered to be alive. An object that the WeakReference type refers to has not been garbage-collected. The WeakReference type can remain alive long after the object to which it referred is gone. An indication of whether this WeakReference object is alive is obtained through the read-only IsAlive property of the WeakReference object. This property returns a bool indicating whether this object is alive (true) or not (false). When an object is not alive or when its key does not exist in the Dictionary<T,WeakReference>, this method creates a new object with the same key as the one passed in to the indexer and adds it to the Dictionary<T,WeakReference>.

The indexer also implements the mechanism to add objects to the cache. This method creates a WeakReference object that will hold a weak reference to your object of type U. Each object of type U in the cache is contained within a WeakReference object. This is the core of the caching mechanism used in this recipe. A WeakReference that references an object (its target) allows that object to later be referenced. When the target of the WeakReference object is also referenced by a strong (i.e., normal) reference, the garbage collector cannot collect the target object. But if no references are made to the target stored in this WeakReference object, the garbage collector can collect this object to make room in the managed heap for new objects.

After creating the WeakReference object, the Dictionary<T,WeakReference> is searched for the same key of type T that you want to add. If an object with that key exists, it is overwritten with the new object; otherwise, the Add method of the Dictionary<T,WeakReference> class is called.

Quite a bit of extra work is required in the calling code to support a cache of heterogeneous objects. More responsibility is placed on the user of this cache object, which can quickly lead to usability and maintenance problems if not written correctly.

The code to exercise the ObjCache<T,U> class is shown in Example 3-20.

Example 3-20. Using the ObjCache class

 // Create the cache here. static ObjCache<string, SomeComplexObj> OC = new ObjCache<string, SomeComplexObj>(); public void TestObjCache() {     OC["ID1"] = new SomeComplexObj();     OC["ID2"] = new SomeComplexObj();     OC["ID3"] = new SomeComplexObj();     OC["ID4"] = new SomeComplexObj();     OC["ID5"] = new SomeComplexObj();     Console.WriteLine("\r\n--> Add 5 weak references");     Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots());     Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache());     //////////////  BEGIN COLLECT  //////////////     GC.Collect();     GC.WaitForPendingFinalizers();     //////////////  END COLLECT  //////////////     Console.WriteLine("\r\n--> Collect all weak references");     Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots());     Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache());     OC["ID1"] = new SomeComplexObj();     ,ch03.10094 Page 194 Thursday, January 5, 2006 12:56 PM     OC["ID2"] = new SomeComplexObj();     OC["ID3"] = new SomeComplexObj();     OC["ID4"] = new SomeComplexObj();     OC["ID5"] = new SomeComplexObj();     Console.WriteLine("\r\n--> Add 5 weak references");     Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots());     Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache());     CreateObjLongMethod();     Create135();     CollectAll(); } private void CreateObjLongMethod() {     Console.WriteLine("\r\n--> Obtain ID1");     string id1 = "ID1";     if (OC.IsObjAlive(ref id1))     {         SomeComplexObj SCOTemp = OC["ID1"];         SCOTemp.IDCode = 100;         Console.WriteLine("SCOTemp.IDCode = " + SCOTemp.IDCode);     }     else     {         Console.WriteLine("Object ID1 does not exist…Creating new ID1…");         OC["ID1"] = new SomeComplexObj());         SomeComplexObj SCOTemp = OC["ID1"];         SCOTemp.IDCode = 101;         Console.WriteLine("SCOTemp.IDCode = " + SCOTemp.IDCode);     } } private void Create135() {     Console.WriteLine("\r\n--> Obtain ID1, ID3, ID5");     SomeComplexObj SCO1 = OC["ID1"];     SomeComplexObj SCO3 = OC["ID3"];     SomeComplexObj SCO5 = OC["ID5"];     SCO1.IDCode = 1000;     SCO3.IDCode = 3000;     SCO5.IDCode = 5000;     ////////////// BEGIN COLLECT //////////////     GC.Collect();     GC.WaitForPendingFinalizers();     ////////////// END COLLECT //////////////     Console.WriteLine("\r\n--> Collect all weak references");     Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots());     Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache());     Console.WriteLine("SCO1.IDCode = " + SCO1.IDCode);     Console.WriteLine("SCO3.IDCode = " + SCO3.IDCode);     Console.WriteLine("SCO5.IDCode = " + SCO5.IDCode);     string id2 = "ID2";     Console.WriteLine("\r\n--> Get ID2, which has been collected. ID2 Exists ==" +                        OC.IsObjAlive(ref id2));     SomeComplexObj SCO2 = OC["ID2"];     Console.WriteLine("ID2 has now been re-created.  ID2 Exists == " + OC.IsObjAlive(ref id2));     Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache());     SCO2.IDCode = 2000;     Console.WriteLine("SCO2.IDCode = " + SCO2.IDCode);     ////////////// BEGIN COLLECT //////////////     GC.Collect();     GC.WaitForPendingFinalizers();     //////////////  END COLLECT  //////////////     Console.WriteLine("\r\n--> Collect all weak references");     Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots());     Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache()); } private void CollectAll() {     ////////////// BEGIN COLLECT //////////////     GC.Collect();     GC.WaitForPendingFinalizers();     //////////////  END COLLECT  //////////////     Console.WriteLine("\r\n--> Collect all weak references");     Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots());     Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache()); }

The output of this test code is shown here:

 --> Add 5 weak references OC.TotalCacheSlots = 5 OC.AliveObjsInCache = 5 --> Collect all weak references OC.TotalCacheSlots = 5 OC.AliveObjsInCache = 0 --> Add 5 weak references OC.TotalCacheSlots = 5 OC.AliveObjsInCache = 5 --> Obtain ID1 SCOTemp.IDCode = 100 --> Obtain ID1, ID3, ID5 --> Collect all weak references OC.TotalCacheSlots = 5 OC.AliveObjsInCache = 3 SCO1.IDCode = 1000 SCO3.IDCode = 3000 SCO5.IDCode = 5000 --> Get ID2, which has been collected. ID2 Exists ==False ID2 has now been re-created. ID2 Exists == True OC.AliveObjsInCache = 4 SCO2.IDCode = 2000 --> Collect all weak references OC.TotalCacheSlots = 5 OC.AliveObjsInCache = 4 --> Collect all weak references OC.TotalCacheSlots = 5 OC.AliveObjsInCache = 0

Discussion

Caching involves storing frequently used objects, particularly those that are expensive to create and re-create, in memory for fast access. This technique is in contrast to recreating these objects through some time-consuming mechanism (e.g., from data in a database or from a file on disk) every time they are needed. By storing frequently used objects such as theseso that you do not have to create them nearly as muchyou can further improve the performance of the application.

When deciding which types of items can be cached, you should look for objects that take a long time to create and/or initialize. For example, if an object's creation involves one or more calls to a database, to a file on disk, or to a network resource, it can be considered as a candidate for caching. In addition to selecting objects with long creation times, these objects should also be frequently used by the application. Selection depends on a combination of the frequency of use and the average time for which it is used in any given usage. Objects that remain in use for a long time when they are retrieved from the cache may work better in this cache than those that are frequently used but for only a very short period of time.

If you do not want to overwrite cached items having the same key as the object you are attempting to insert into the cache, the set accessor of the indexer must be modified. The code for the set accessor could be modified to this:

 public U this[T key] {     get     {         if (!cache.ContainsKey(key) || !IsObjAlive(ref key))         {             this[key] = new U();         }         return ((U)((WeakReference)cache[key]).Target); } set {     WeakReference WR = new WeakReference(value, false);     if (cache.ContainsKey(key))     {         cache[key] = WR;     }     else     {         cache.Add(key, WR);     } } }

You could also add a mechanism to calculate the cache-hit ratio for this cache. The cache-hit ratio is the ratio of hitsevery time an existing object is requested from the Dictionary<T,WeakReference>to the total number of calls made to attempt a retrieval of an object. This can give you a good indication of how well your ObjCache<T,U> is working. The code to add to this class to implement calculation of a cache-hit ratio is shown highlighted in Example 3-21.

Example 3-21. Calculating a cache-hit ratio

  private float numberOfGets = 0; private float numberOfHits = 0; public float HitMissRatioPercent() {     if (numberOfGets == 0)     {         v return (0);     }     else     {         return ((numberOfHits / numberOfGets) * 100);     } } public U this[T key] {     get     {          ++numberOfGets;         if (!cache.ContainsKey(key) || !IsObjAlive(ref key))         {             this[key] = new U();         }                  else         {             ++numberOfHits;         }         return ((U)((WeakReference)cache[key]).Target);     }     set     {         WeakReference WR = new WeakReference(value, false);         if (cache.ContainsKey(key))         {             cache[key] = WR;         }         else         {             cache.Add(key, WR);         }     } }

The numberOfGets field tracks the number of calls made to the get accessor of the indexer. The numberOfHits field tracks the number of times that an object to be retrieved exists in the cache. The HitMissRatioPercent method returns the numberOfHits divided by the numberOfGets as a percentage. The higher the percent, the better your cache is operating (100 percent is equal to a hit every time the get accessor of the indexer is called). A lower percentage indicates that this cache object is not working efficiently (zero percent is equal to a miss every time the get accessor of the indexer is called). A very low percentage indicates that the cache object may not be the correct solution to your problem or that you are not caching the correct object(s).

The WeakReference objects created for the ObjCache<T,U> class do not track objects after they are finalized. This would add much more complexity than is needed by this class.

Remember, a caching scheme adds complexity to your application. The most a caching scheme can do for your application is to enhance performance and possibly place less stress on memory resources. You should consider this when deciding whether to implement a caching scheme such as the one in this recipe.