Derby
  1. Derby
  2. DERBY-3734

Maximum value allowed for derby.storage.fileCacheSize (100) is too low for large system. Increase the maximum value and redocument the property.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.3.3.0, 10.4.1.3
    • Fix Version/s: 10.4.2.0, 10.5.1.1
    • Component/s: None
    • Labels:
      None
    • Environment:
      Derby 10.3.3
    • Urgency:
      Urgent
    • Bug behavior facts:
      Performance

      Description

      Increasing the value of the undocumented property: derby.storage.fileCacheSize improves Derby performance in our system but the maximum allowed value (100) is not large enough to accommodate our system. Our performance engineer reports:

      The following stack shows items being evicted from Derby's container cache. Extra debug code showed that the cache was at its maximum size (100), and that about 1 in 25 accesses to the cache was resulting in a miss (forcing another item to be evicted). Since this results in synchronous disk write, performance is bottlenecked on IO. Patching Derby to allow the cache to grow to 200 entries solved the performance problem.

      Performance is severely degraded. CPU utilization is low – performance is IO bound. A sampling of stack dumps for the key thread consistently have the following methods at the top of the stack:

      at sun/nio/ch/FileChannelImpl.force0(Native Method)
      at sun/nio/ch/FileChannelImpl.force(FileChannelImpl.java:392(Compiled Code))
      at org/apache/derby/impl/io/DirRandomAccessFile4.sync(Bytecode PC:5(Compiled Code))
      at org/apache/derby/impl/store/raw/data/RAFContainer.writeRAFHeader(Bytecode PC:86(Compiled Code))
      at org/apache/derby/impl/store/raw/data/RAFContainer.clean(Bytecode PC:84(Compiled Code))
      at org/apache/derby/impl/services/cache/CachedItem.clean(Bytecode PC:7(Compiled Code))
      at org/apache/derby/impl/services/cache/Clock.rotateClock(Bytecode PC:7(Compiled Code))
      at org/apache/derby/impl/services/cache/Clock.findFreeItem(Bytecode PC:17(Compiled Code))
      at org/apache/derby/impl/services/cache/Clock.find(Bytecode PC:71(Compiled Code))
      at org/apache/derby/impl/store/raw/data/BaseDataFileFactory.openContainer(Bytecode PC:65(Compiled Code))
      at org/apache/derby/impl/store/raw/data/BaseDataFileFactory.openContainer(Bytecode PC:7(Compiled Code))
      at org/apache/derby/impl/store/raw/xact/Xact.openContainer(Bytecode PC:29(Compiled Code))

        Activity

        Hide
        Knut Anders Hatlen added a comment -

        +1

        I don't think there's any need to limit the maximum value for this setting. On some systems, you may exceed the maximum allowed number of open files if you increase the size of the container cache too much, but that's not something we need to worry about if we leave the default as it is today.

        Show
        Knut Anders Hatlen added a comment - +1 I don't think there's any need to limit the maximum value for this setting. On some systems, you may exceed the maximum allowed number of open files if you increase the size of the container cache too much, but that's not something we need to worry about if we leave the default as it is today.
        Hide
        Mike Matrigali added a comment -

        unless there are any other opinions I am planning on just upping the maximum to max int and leave the default values as they are.

        Show
        Mike Matrigali added a comment - unless there are any other opinions I am planning on just upping the maximum to max int and leave the default values as they are.
        Hide
        Mike Matrigali added a comment -

        here is a patch of the simple change. I only plan to hand test the parameters. No default behavior is expected to change.

        Show
        Mike Matrigali added a comment - here is a patch of the simple change. I only plan to hand test the parameters. No default behavior is expected to change.
        Hide
        Mike Matrigali added a comment -

        The fix for this has been checked into trunk, 10.3 branch and 10.4 branch.

        Show
        Mike Matrigali added a comment - The fix for this has been checked into trunk, 10.3 branch and 10.4 branch.
        Hide
        Mike Matrigali added a comment -

        Fix to unlimit the maximum setting for the container cache have been applied to trunk, 10.3 branch and 10.4 branch.

        Show
        Mike Matrigali added a comment - Fix to unlimit the maximum setting for the container cache have been applied to trunk, 10.3 branch and 10.4 branch.

          People

          • Assignee:
            Mike Matrigali
            Reporter:
            Stan Bradbury
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development