Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15308

Create ACID table failed intermittently: due to Postgres (SQLState=25P02, ErrorCode=0)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.2.0
    • Component/s: Transactions
    • Labels:
      None
    • Target Version/s:

      Description

      if 2 concurrent calls to MutexApi.acquireLock() happen with the same "key" value and there is no row in AUX_TABLE for that value yet (i.e. both are attempting to insert it) Postgres kills the txn which gets the Duplicate Key error and no more statements can be executed on this txn.
      (This is different from the way most DBs behave).

      Caused by: MetaException(message:Unable to lock 'CheckLock' due to: ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLState=25P02, ErrorCode=0); org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block
      	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)
      	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)
      	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
      	at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:561)
      	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:405)
      	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:285)
      	at com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.acquireLock(TxnHandler.java:3250)
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:2319)
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1022)
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:794)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:5941)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)
      	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
      	at com.sun.proxy.$Proxy30.lock(Unknown Source)
      	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:2109)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:154)
      	at com.sun.proxy.$Proxy31.lock(Unknown Source)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2259)
      	at com.sun.proxy.$Proxy31.lock(Unknown Source)
      	at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager$SynchronizedMetaStoreClient.lock(DbTxnManager.java:745)
      	at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:103)
      	at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:341)
      	at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocksWithHeartbeatDelay(DbTxnManager.java:357)
      	at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:167)
      	at org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:985)
      	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1321)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1095)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1088)
      	at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242)
      	at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
      	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:334)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
      	at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:347)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      )
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.acquireLock(TxnHandler.java:3268)
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:2319)
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1022)
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:794)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:5941)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      

        Attachments

        1. HIVE-15308.01.patch
          0.9 kB
          Eugene Koifman

          Activity

            People

            • Assignee:
              ekoifman Eugene Koifman
              Reporter:
              ekoifman Eugene Koifman
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: