Details

    • Urgency:
      Normal
    • Issue & fix info:
      Known fix, Workaround attached
    • Bug behavior facts:
      Crash

      Description

      When using JTA for transaction control and a transaction timeout is set,
      EmbedXAResource ends up calling XATransactionState.scheduleTimeoutTask() which
      in turn registers a timeoutTask with java.util.Timer. In the normal case where
      the transaction finishes before the timeout, XATransactionState.xa_finalize()
      then calls timeoutTask.cancel(). So far this so good. The problem, however, is
      that java.util.TimerTask.cancel() does not actually remove the task from the
      timer queue, meaning that a strong reference to the timeoutTask is kept (and
      through that to XATransactionState, the EmbedConnection, etc). The reference
      is not removed until the time at which the timeout would have fired, which can
      be a long time. Under load this can quickly lead to an OOM situation.

      A simple fix is to call Timer.purge() every so often. While the javadocs talk
      about purge() being rarely needed and that it's not extremely cheap, I've
      found that calling it after every cancel() is the best approach, for several
      reasons: 1) the scenario here is that almost all tasks are cancelled, and
      hence this somewhat fits the Timer.purge() description of an "application that
      cancels a large number of tasks"; 2) there usually isn't a very large number
      of simultaneous transactions, and hence purge() is actually quite cheap; 3)
      this ensures the strong reference is immediately removed, allowing the GC to
      do a better job. Interestingly enough, I've had this exact same issue on a
      different type of db, and I had tested the purge() there and found it to be in
      the sub-microsecond range for 100 transactions (or similar - I don't recall
      the exact data), i.e. completely negligible.

      In short, my suggestion is to change xa_finalize as follows:

      synchronized void xa_finalize() {
      if (timeoutTask != null)

      { timeoutTask.cancel(); Monitor.getMonitor().getTimerFactory(). getCancellationTimer().purge(); }

      isFinished = true;
      }

      As a temporary workaround, applications can do this themselves, i.e.
      add something like the following whenever they close a Connection:

      import org.apache.derby.iapi.services.monitor.Monitor;
      Monitor.getMonitor().getTimerFactory().getCancellationTimer().purge();

      1. derby-4137-2a-reduce_memory_footprint.diff
        7 kB
        Kristian Waagan
      2. derby-4137-1b-purge_on_cancel.diff
        8 kB
        Kristian Waagan
      3. derby-4137-1a-purge_on_cancel.diff
        7 kB
        Kristian Waagan
      4. derby-4137-1a-purge_on_cancel.stat
        0.2 kB
        Kristian Waagan

        Issue Links

          Activity

          Knut Anders Hatlen made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Workflow jira [ 12459661 ] Default workflow, editable Closed status [ 12802501 ]
          Knut Anders Hatlen made changes -
          Link This issue is related to DERBY-6114 [ DERBY-6114 ]
          Myrna van Lunteren made changes -
          Fix Version/s 10.8.2.2 [ 12317968 ]
          Fix Version/s 10.8.2.0 [ 12317955 ]
          Myrna van Lunteren made changes -
          Fix Version/s 10.8.2.0 [ 12317955 ]
          Fix Version/s 10.8.1.6 [ 12316676 ]
          Knut Anders Hatlen made changes -
          Link This issue is related to DERBY-5375 [ DERBY-5375 ]
          Kristian Waagan made changes -
          Fix Version/s 10.5.3.2 [ 12315436 ]
          Fix Version/s 10.6.2.3 [ 12315434 ]
          Fix Version/s 10.7.1.4 [ 12315902 ]
          Fix Version/s 10.8.1.6 [ 12316676 ]
          Knut Anders Hatlen made changes -
          Link This issue breaks DERBY-5291 [ DERBY-5291 ]
          Kristian Waagan made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Issue & fix info [Known fix, Patch Available, Workaround attached] [Known fix, Workaround attached]
          Fix Version/s 10.9.0.0 [ 12316344 ]
          Resolution Fixed [ 1 ]
          Kristian Waagan made changes -
          Kristian Waagan made changes -
          Attachment derby-4137-1b-purge_on_cancel.diff [ 12481787 ]
          Kristian Waagan made changes -
          Issue & fix info [Known fix, Workaround attached] [Known fix, Patch Available, Workaround attached]
          Kristian Waagan made changes -
          Link This issue incorporates DERBY-5264 [ DERBY-5264 ]
          Kristian Waagan made changes -
          Attachment derby-4137-1a-purge_on_cancel.stat [ 12481662 ]
          Attachment derby-4137-1a-purge_on_cancel.diff [ 12481663 ]
          Kristian Waagan made changes -
          Assignee Kristian Waagan [ kristwaa ]
          Kathey Marsden made changes -
          Labels derby_triage10_5_2
          Kristian Waagan made changes -
          Issue & fix info [Patch Available] [Known fix, Workaround attached]
          Rick Hillegas made changes -
          Field Original Value New Value
          Bug behavior facts [Crash]
          Urgency Normal
          Ronald Tschalaer created issue -

            People

            • Assignee:
              Kristian Waagan
              Reporter:
              Ronald Tschalaer
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development