Derby
  1. Derby
  2. DERBY-3368

Threading issue with DependencyManager may cause in-memory dependencies to be lost.

    Details

    • Urgency:
      Normal
    • Bug behavior facts:
      Wrong query result

      Description

      When a thread compiles a language prepared statement P a set of in-memory Dependency objects is created, e.g. if P accesses table A

      Dependency

      {P depends on A}

      When this Dependency is added to the dependency manager if an equivalent one (same provider and dependent) exists then the duplicate will not be added.

      The StatementContext keeps track of these added Dependency so that if the compilation fails the Dependency will be removed, comparing by the exact same Dependency object (not by equivalence).

      If a thread T1 compiling P fails, then another thread may try to compile P (same object). If this second thread T2 compiles successfully the following could happen:

      1) T1 compiles P creates Dependency {P depends on A}

      in dependency manager
      2) T1 fails to compile, but does not yet execute its cleanup
      3) T2 compiles P successfully, attempts to add Dependency

      {P depends on A} to the manager but it is a duplicate so T1's version is left and T2's is not added.
      4) T1 completes its cleanup and removes Dependency {P depends on A}

      5) P no longer depends on A

      Concern is that the security system GRANT/REVOKE is based upon the dependency manager as well as correctness for indexes (e.g. this could cause a recompile to be missed for an INSERT table when an index is added).

      For this to actually happen there has to be a situation where one thread (connection) can compile a statement that another one fails on (and be compiling at near identical times). I haven't got a reproducible case yet, but I can get two statements to be compiling the same statement plan (P).

        Issue Links

          Activity

          Hide
          Kathey Marsden added a comment -

          Taking off of High Value Fix until we have a reproduction. Adding wrong results which could occur with this issue.

          Show
          Kathey Marsden added a comment - Taking off of High Value Fix until we have a reproduction. Adding wrong results which could occur with this issue.
          Hide
          Tiago R. Espinha added a comment -

          Triaged for 10.5.2.

          Assigned Normal urgency.

          Show
          Tiago R. Espinha added a comment - Triaged for 10.5.2. Assigned Normal urgency.
          Hide
          Kathey Marsden added a comment -

          This could potentially result in a corrupt index and wrong results so marking as a high value fix candidate.

          Show
          Kathey Marsden added a comment - This could potentially result in a corrupt index and wrong results so marking as a high value fix candidate.
          Hide
          Daniel John Debrunner added a comment -

          Came across this while trying to understand the dependency manager code for DERBY-3102 and DERBY-2380

          Show
          Daniel John Debrunner added a comment - Came across this while trying to understand the dependency manager code for DERBY-3102 and DERBY-2380
          Hide
          Daniel John Debrunner added a comment -

          Another find is that CompileContextImpl also has code to clear the dependencies on a language prepared statement when a compile fails, though it will not always be executed due to the inconsistent popping & pushing of compiler contexts.

          The role of clearing in-memory dependencies for a failed compile seems more suited to the compiler context than the statement context, at least not both should be trying to do it.

          Show
          Daniel John Debrunner added a comment - Another find is that CompileContextImpl also has code to clear the dependencies on a language prepared statement when a compile fails, though it will not always be executed due to the inconsistent popping & pushing of compiler contexts. The role of clearing in-memory dependencies for a failed compile seems more suited to the compiler context than the statement context, at least not both should be trying to do it.
          Hide
          Daniel John Debrunner added a comment -

          As an aside one strange aspect is that if the Dependency is not added to the manager due to an existing duplicate then it is still added to the list in the StatementContext. This is pointless as the Dependency will not exist in the manager and the purpose of the list in the statement context is to remove dependencies on error.

          Show
          Daniel John Debrunner added a comment - As an aside one strange aspect is that if the Dependency is not added to the manager due to an existing duplicate then it is still added to the list in the StatementContext. This is pointless as the Dependency will not exist in the manager and the purpose of the list in the statement context is to remove dependencies on error.

            People

            • Assignee:
              Unassigned
              Reporter:
              Daniel John Debrunner
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development