Virtual Directory Caching

So there's been a lot of discussion lately about Virtual Directories and caching, so here's my take on it.

First of all, the key concept behind a Virtual Directory is that the virtualized data is not be moved from the back end repositories and should be accessed in real time as the Virtual Directory is queried by the LDAP client.

So in my humble opinion... The Virtual Directory must not rely on a cache when executing search operations. That is to say that caching mechanisms, when utilized, must be optional and not be a core mechanism in the virtualization process. When caching is needed in a particular scenario it should utilize a dynamic base where data remains for a specified time. I believe that caching mechanisms detract from the true nature of a Virtual Directory. The point of a Virtual Directory is that it is just that, Virtual. One cannot point to any specific place and say “This is where the Virtual Directory data exists”. When any type of cache is implemented this is no longer the place as there is now a presence to the Virtualized data. To minimize the “presence factor”, attributes held within the cache should have a definable duration or Time to Live (TTL). When the TTL expires the information should be flushed from the cache. This reduces the risk of data being improperly obtained from the cache when queried. Constantly changing data must be have a short TTL so that the most recent values are present, a must when the Virtual Directory is used for Authorization or Authentication. Less volatile data could have a longer TTL and potentially improve Virtual Directory performance. In particular, static, or file-based caches offer additional issues in the form of increased latency when executing and returning search queries. When the cache is held in a non-static form (i.e., Server RAM) latency delays drop and there is less of a chance that the information held in the cache can be accessed by other people or applications.

Of course, that's not to say that caching is entirely bad, but I'll leave that for a future entry along with an examination of the political factors involved with a Virtual Directory.