Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9938

Add retry logic to DbTxnMgr instead of aborting transactions.

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.14.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Sometimes parallel updates using DBTxnMgr results in the following error trace

      5325 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver> 
      5351 [main] ERROR org.apache.hadoop.hive.ql.Driver - FAILED: Error in acquiring locks: Error communicating with the metastore 
      org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastore 
      at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:100) 
      at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:194) 
      

      Internally looking at the postgres logs we see

      2015-02-02 06:36:05,632 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: org.apache.thrift.TException: MetaException(message:Unable to update transaction database org.postgresql.util.PSQLException: ERROR: could not serialize access due to concurrent update 
      
      

      Ideally we should add a retry logic to retry the failed transaction.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                bharathv Bharath Vissapragada
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: