Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9938

Add retry logic to DbTxnMgr instead of aborting transactions.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.14.0
    • None
    • None
    • None

    Description

      Sometimes parallel updates using DBTxnMgr results in the following error trace

      5325 [main] INFO org.apache.hadoop.hive.ql.log.PerfLogger - <PERFLOG method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver> 
      5351 [main] ERROR org.apache.hadoop.hive.ql.Driver - FAILED: Error in acquiring locks: Error communicating with the metastore 
      org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastore 
      at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:100) 
      at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:194) 
      

      Internally looking at the postgres logs we see

      2015-02-02 06:36:05,632 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: org.apache.thrift.TException: MetaException(message:Unable to update transaction database org.postgresql.util.PSQLException: ERROR: could not serialize access due to concurrent update 
      
      

      Ideally we should add a retry logic to retry the failed transaction.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              bharathv Bharath Vissapragada
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: