Derby
  1. Derby
  2. DERBY-4055

Space may not be reclaimed if row locks are not available after three retries

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.1.1
    • Fix Version/s: None
    • Component/s: Store
    • Urgency:
      Normal
    • Issue & fix info:
      High Value Fix, Workaround attached

      Description

      In a multithreaded clob update where the same row is being updated, space will not be reclaimed. The offending code is in ReclaimSpaceHelper:

      RecordHandle headRecord = work.getHeadRowHandle();

      if (!container_rlock.lockRecordForWrite(
      tran, headRecord, false /* not insert /, false / nowait */))
      {
      // cannot get the row lock, retry
      tran.abort();
      if (work.incrAttempts() < 3)

      { return Serviceable.REQUEUE; }
      else
      {
      // If code gets here, the space will be lost forever, and
      // can only be reclaimed by a full offline compress of the
      // table/index.

      if (SanityManager.DEBUG)
      {
      if (SanityManager.DEBUG_ON(DaemonService.DaemonTrace))
      { SanityManager.DEBUG( DaemonService.DaemonTrace, " gave up after 3 tries to get row lock " + work); }
      }
      return Serviceable.DONE;
      }
      }


      If we cannot get the lock after three tries we give up. The reproduction for this issue is in the test store.ClobReclamationTest.xtestMultiThreadUpdateSingleRow().


      This issue also used to reference the code below and has some references to trying to get a reproduction for that issue, but that work has moved to DERBY-4054. Please see DERBY-4054 for work on the container lock issue.

      ContainerHandle containerHdl =
      openContainerNW(tran, container_rlock, work.getContainerId());

      if (containerHdl == null)
      {
      tran.abort();

      if (SanityManager.DEBUG)
      {
      if (SanityManager.DEBUG_ON(DaemonService.DaemonTrace))
      { SanityManager.DEBUG( DaemonService.DaemonTrace, " aborted " + work + " because container is locked or dropped"); }
      }

      if (work.incrAttempts() < 3) // retry this for serveral times
      { return Serviceable.REQUEUE; }

      else
      {
      // If code gets here, the space will be lost forever, and
      // can only be reclaimed by a full offline compress of the
      // table/index.

      if (SanityManager.DEBUG)
      {
      if (SanityManager.DEBUG_ON(DaemonService.DaemonTrace))

      { SanityManager.DEBUG( DaemonService.DaemonTrace, " gave up after 3 tries to get container lock " + work); }

      }

      return Serviceable.DONE;
      }
      }

        Issue Links

          Activity

          Kathey Marsden created issue -
          Kathey Marsden made changes -
          Field Original Value New Value
          Priority Minor [ 4 ] Major [ 3 ]
          Kathey Marsden made changes -
          Link This issue is related to DERBY-4050 [ DERBY-4050 ]
          Kathey Marsden made changes -
          Attachment derby.log.T_RawStoreFactoryWithAssert [ 12400145 ]
          Kathey Marsden made changes -
          Summary Space may not be reclaimed if locks are not available after three retries (code inspection) Space may not be reclaimed if row locks are not available after three retries
          Description I don't have a reproduction for these cases but there are two places in ReclaimSpaceHelper where reclamation will give up after three tries if it cannot obtain the lock to reclaim the space. The first code path is:
          ContainerHandle containerHdl =
          openContainerNW(tran, container_rlock, work.getContainerId());

          if (containerHdl == null)
          {
          tran.abort();

          if (SanityManager.DEBUG)
                      {
                          if (SanityManager.DEBUG_ON(DaemonService.DaemonTrace))
                          {
                              SanityManager.DEBUG(
                                  DaemonService.DaemonTrace, " aborted " + work +
                                  " because container is locked or dropped");
                          }
                      }

          if (work.incrAttempts() < 3) // retry this for serveral times
                      {
          return Serviceable.REQUEUE;
                      }
          else
                      {
                          // If code gets here, the space will be lost forever, and
                          // can only be reclaimed by a full offline compress of the
                          // table/index.

                          if (SanityManager.DEBUG)
                          {
                              if (SanityManager.DEBUG_ON(DaemonService.DaemonTrace))
                              {
                                  SanityManager.DEBUG(
                                      DaemonService.DaemonTrace,
                                      " gave up after 3 tries to get container lock " +
                                      work);
                              }
                          }

          return Serviceable.DONE;
                      }
          }

          the second is:
          RecordHandle headRecord = work.getHeadRowHandle();

          if (!container_rlock.lockRecordForWrite(
                          tran, headRecord, false /* not insert */, false /* nowait */))
          {
          // cannot get the row lock, retry
          tran.abort();
          if (work.incrAttempts() < 3)
                      {
          return Serviceable.REQUEUE;
                      }
          else
                      {
                          // If code gets here, the space will be lost forever, and
                          // can only be reclaimed by a full offline compress of the
                          // table/index.

                          if (SanityManager.DEBUG)
                          {
                              if (SanityManager.DEBUG_ON(DaemonService.DaemonTrace))
                              {
                                  SanityManager.DEBUG(
                                      DaemonService.DaemonTrace,
                                      " gave up after 3 tries to get row lock " +
                                      work);
                              }
                          }
          return Serviceable.DONE;
                      }
          }

          If working to get a reproduction for these cases, you can set derby.debug.true=DaemonTrace and look for "gave up" in the derby.log.


          In a multithreaded clob update where the same row is being updated, space will not be reclaimed. The offending code is in ReclaimSpaceHelper:

          RecordHandle headRecord = work.getHeadRowHandle();

          if (!container_rlock.lockRecordForWrite(
                          tran, headRecord, false /* not insert */, false /* nowait */))
          {
          // cannot get the row lock, retry
          tran.abort();
          if (work.incrAttempts() < 3)
                      {
          return Serviceable.REQUEUE;
                      }
          else
                      {
                          // If code gets here, the space will be lost forever, and
                          // can only be reclaimed by a full offline compress of the
                          // table/index.

                          if (SanityManager.DEBUG)
                          {
                              if (SanityManager.DEBUG_ON(DaemonService.DaemonTrace))
                              {
                                  SanityManager.DEBUG(
                                      DaemonService.DaemonTrace,
                                      " gave up after 3 tries to get row lock " +
                                      work);
                              }
                          }
          return Serviceable.DONE;
                      }
          }


          If we cannot get the lock after three tries we give up. The reproduction for this issue is in the test store.ClobReclamationTest.xtestMultiThreadUpdateSingleRow().


          This issue also used to reference the code below and has some references to trying to get a reproduction for that issue, but that work has moved to DERBY-4054. Please see DERBY-4054 for work on the container lock issue.

          ContainerHandle containerHdl =
          openContainerNW(tran, container_rlock, work.getContainerId());

          if (containerHdl == null)
          {
          tran.abort();

          if (SanityManager.DEBUG)
                      {
                          if (SanityManager.DEBUG_ON(DaemonService.DaemonTrace))
                          {
                              SanityManager.DEBUG(
                                  DaemonService.DaemonTrace, " aborted " + work +
                                  " because container is locked or dropped");
                          }
                      }

          if (work.incrAttempts() < 3) // retry this for serveral times
                      {
          return Serviceable.REQUEUE;
                      }
          else
                      {
                          // If code gets here, the space will be lost forever, and
                          // can only be reclaimed by a full offline compress of the
                          // table/index.

                          if (SanityManager.DEBUG)
                          {
                              if (SanityManager.DEBUG_ON(DaemonService.DaemonTrace))
                              {
                                  SanityManager.DEBUG(
                                      DaemonService.DaemonTrace,
                                      " gave up after 3 tries to get container lock " +
                                      work);
                              }
                          }

          return Serviceable.DONE;
                      }
          }


          Kathey Marsden made changes -
          Derby Categories [High Value Fix]
          Myrna van Lunteren made changes -
          Affects Version/s 10.5.1.1 [ 12313771 ]
          Affects Version/s 10.5.0.0 [ 12313010 ]
          Dag H. Wanvik made changes -
          Issue & fix info [High Value Fix]
          Knut Anders Hatlen made changes -
          Urgency Normal
          Issue & fix info [High Value Fix] [High Value Fix, Workaround attached]
          Kathey Marsden made changes -
          Labels derby_triage10_5_2
          Kathey Marsden made changes -
          Link This issue is related to DERBY-5356 [ DERBY-5356 ]
          Mike Matrigali made changes -
          Labels derby_triage10_5_2 derby_triage10_5_2 derby_triage10_9
          Rick Hillegas made changes -
          Link This issue is related to DERBY-5876 [ DERBY-5876 ]
          Kathey Marsden made changes -
          Labels derby_triage10_5_2 derby_triage10_9 derby_triage10_11
          Gavin made changes -
          Workflow jira [ 12452509 ] Default workflow, editable Closed status [ 12802571 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Kathey Marsden
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development