Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10422

EXPLAIN statements leak ACID transactions and locks

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 4.0.0
    • None
    • None
    • ghx-label-4

    Description

      Currently EXPLAIN statements might open ACID transactions and create locks on ACID tables.

      This is not necessary since we won't modify the table. But the real problem is that these transactions and locks are leaked and open forever. They are even getting heartbeated while the coordinator is still running.

      The solution should be to not consume any ACID resources for EXPLAIN statements.

      Reproduction:

      create table test (i int) tblproperties('transactional'='true', 'transactional_properties'='insert_only');
      explain insert overwrite test values (1);
      insert overwrite test values (1);
      ERROR: TransactionException: Failed to acquire lock for transaction 1595
      

      Workarounds when dealing with this issue:

      • Restart the impala cluster so the heartbeating stops. After a while the Hive AcidHouseKeeperService should cleanup the resources (abort the transaction and clear the locks)
      • Manually abort the open transaction via Hive with the ABORT TRANSACTIONS <transaction-id>; command. One can find the transaction id with the help of the SHOW LOCKS and SHOW TRANSACTIONS Hive commands.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            boroknagyz Zoltán Borók-Nagy
            boroknagyz Zoltán Borók-Nagy
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment