Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21871

Multi-statement transactions in direct SQL

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      To access metastore, we may bypass the JDO layer and query the metastore RDBMS directly (we refer to this as direct SQL path).

      There are some methods in Hive metastore that may issue multiple queries against RDBMS to build the return objects (e.g. get_partitions_by_names). Currently going through direct SQL may issue each query to the RDBMS in a different transaction (while afaik going through JDO will create a single transaction to retrieve and compose such objects). This may lead to failures while running some operations concurrently, e.g., in the example above, if a partition is being dropped and partitions are being retrieved using direct SQL path.

      A solution would be to execute all statements needed to retrieve the results for such a function within a single transaction when we use direct SQL path.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jcamachorodriguez Jesus Camacho Rodriguez
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: