Commons JCS
  1. Commons JCS
  2. JCS-15

Removing entries using a name group does not remove them from the disk cache.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: jcs-1.2.7.9.2
    • Fix Version/s: None
    • Component/s: Indexed Disk Cache
    • Labels:
      None

      Description

      The scenario is that I add entries to the cache using name groups, e.g. using keys "0:key0", "0:key1", "1:key0", etc. then issue a remove using a name group, e.g. "0:".

      If all the entries are in memory the remove succeeds. However if some or all have been spooled out to disk those that have been spooled out do not get removed from the cache. The code fragment and cache.ccf at the end of this post show the problem. I'm looking at the source checked out of trunk this morning and IndexedDiskCache.java last changed on 2006-09-22 with revision id 449025 (I originally spotted this issue in release 1.2.7.9.2). Two problems that are immediately obvious are:

      1. The lookup on line 660 always returns null unless I have an item in the cache with key exactly equal to "0:" (which I don't). I think it should be "keyHash.get( k )"
      2. The iter.remove() on line 662 appears to be operating on a copy of the keys.

      I've not tested it, but it looks like the else-if block for GroupId keys suffers the same problems. Also if optimization is enabled for the disk file I suspect data corruption will occur eventually, but again I've not tested this.

      Unfortunately this has stopped us using JCS because we rely on this functionality working since our cache is too big to fit entirely in memory.

      try
      {
      JCS cache = JCS.getInstance( "GeneralCache" );

      for ( int ii = 0; ii < 100; ++ii )
      {
      for ( int jj = 0; jj < 10; ++jj )

      { final String k = jj + ":key" + ii; final String d = "Some data for key " + k; cache.put( k, d ); }

      }
      System.out.println( cache.get( "0:key0" ) );
      cache.remove( "0:" );
      // Next line should print "null"
      System.out.println( cache.get( "0:key0" ) );
      }
      catch ( CacheException e )
      {
      e.printStackTrace();
      }

      try
      {
      JCS cache = JCS.getInstance( "GeneralCache2" );

      for ( int ii = 0; ii < 100; ++ii )
      {
      for ( int jj = 0; jj < 10; ++jj )

      { final String k = jj + ":key" + ii; final String d = "Some data for key " + k; cache.put( k, d ); }

      }
      System.out.println( cache.get( "0:key0" ) );
      cache.remove( "0:" );
      // Next line should print "null"
      System.out.println( cache.get( "0:key0" ) );
      }
      catch ( CacheException e )
      {
      e.printStackTrace();
      }

      The contents of cache.ccf are:

      1. DEFAULT CACHE REGION

      jcs.default=DC
      jcs.default.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
      jcs.default.cacheattributes.MaxObjects=1
      jcs.default.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache

      1. PRE-DEFINED CACHE REGIONS

      jcs.region.GeneralCache=DC
      jcs.region.GeneralCache.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
      jcs.region.GeneralCache.cacheattributes.MaxObjects=1
      jcs.region.GeneralCache.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
      jcs.region.GeneralCache.cacheattributes.UseMemoryShrinker=true
      jcs.region.GeneralCache.cacheattributes.MaxMemoryIdleTimeSeconds=3600
      jcs.region.GeneralCache.cacheattributes.ShrinkerIntervalSeconds=60
      jcs.region.GeneralCache.cacheattributes.MaxSpoolPerRun=500
      jcs.region.GeneralCache.elementattributes=org.apache.jcs.engine.ElementAttributes
      jcs.region.GeneralCache.elementattributes.IsEternal=false

      jcs.region.GeneralCache2=
      jcs.region.GeneralCache2.cacheattributes=org.apache.jcs.engine.CompositeCacheAttributes
      jcs.region.GeneralCache2.cacheattributes.MaxObjects=2000
      jcs.region.GeneralCache2.cacheattributes.MemoryCacheName=org.apache.jcs.engine.memory.lru.LRUMemoryCache
      jcs.region.GeneralCache2.cacheattributes.UseMemoryShrinker=true
      jcs.region.GeneralCache2.cacheattributes.MaxMemoryIdleTimeSeconds=3600
      jcs.region.GeneralCache2.cacheattributes.ShrinkerIntervalSeconds=60
      jcs.region.GeneralCache2.cacheattributes.MaxSpoolPerRun=500
      jcs.region.GeneralCache2.elementattributes=org.apache.jcs.engine.ElementAttributes
      jcs.region.GeneralCache2.elementattributes.IsEternal=false

      1. AVAILABLE AUXILIARY CACHES
        jcs.auxiliary.DC=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheFactory
        jcs.auxiliary.DC.attributes=org.apache.jcs.auxiliary.disk.indexed.IndexedDiskCacheAttributes
        jcs.auxiliary.DC.attributes.DiskPath=cache-files
        jcs.auxiliary.DC.attributes.maxKeySize=100000

        Activity

        Hide
        Aaron Smuts added a comment -

        So the problem is that purgatory partial key removal isn't working, I guess. I'll look into it.

        I'm not sure how I feel about partial key removal . . .

        Show
        Aaron Smuts added a comment - So the problem is that purgatory partial key removal isn't working, I guess. I'll look into it. I'm not sure how I feel about partial key removal . . .
        Hide
        Aaron Smuts added a comment -

        I understand the problem. When using the internal lru for key storage the disk cache cannot do partial removes properly. It's remving from an interator that is a copy of the underlying LRU. Hence, it has no effect.

        I will either change the iterator, or change the remove call for partial keys.

        Show
        Aaron Smuts added a comment - I understand the problem. When using the internal lru for key storage the disk cache cannot do partial removes properly. It's remving from an interator that is a copy of the underlying LRU. Hence, it has no effect. I will either change the iterator, or change the remove call for partial keys.
        Hide
        Aaron Smuts added a comment -

        Fixed and in SVN.

        I changed the remove routine to build up a list and then to remove these one by one.

        Show
        Aaron Smuts added a comment - Fixed and in SVN. I changed the remove routine to build up a list and then to remove these one by one.

          People

          • Assignee:
            Aaron Smuts
            Reporter:
            Kevin Preece
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development