Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-84

MetaStore Client is not thread safe

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.6.0
    • 0.3.0
    • Metastore
    • None
    • with patch for hive-77 - run:

      ant -lib ./testlibs -Dtestcase=TestMTQueries test

    • Reviewed

    Description

      when running DDL Tasks in concurrent threads - the following exception trace is observed:

      java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
      at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:207)
      at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:209)
      at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174)
      at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:185)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:210)
      at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:390)
      at org.apache.hadoop.hive.ql.QTestUtil$QTRunner.run(QTestUtil.java:681)
      at java.lang.Thread.run(Thread.java:619)
      Caused by: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MTable@3bc8d400" us\ ing statement "INSERT INTO TBLS (TBL_ID,CREATE_TIME,DB_ID,RETENTION,TBL_NAME,SD_ID,OWNER,LAST_ACCESS_TIME) VALUES (?,?,?\ ,?,?,?,?,?)" failed : The statement was aborted because it would have caused a duplicate key value in a unique or primar\ y key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
      NestedThrowables:
      java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\ y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
      at org.jpox.jdo.JPOXJDOHelper.getJDOExceptionForJPOXException(JPOXJDOHelper.java:291)
      at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:671)
      at org.jpox.jdo.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:691)
      at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:479)
      at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:292)
      at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:252)
      at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:205)
      ... 7 more
      Caused by: java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a d\ uplicate key value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
      at org.jpox.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:396)
      at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:370)
      at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:157)
      at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:136)
      at org.jpox.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:3082)
      at org.jpox.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:3062)
      at org.jpox.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1231)
      at org.jpox.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1077)
      at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:666)
      ... 12 more
      Caused by: java.sql.SQLException: The statement was aborted because it would have caused a duplicate key value in a uniq\ ue or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      ... 30 more
      Caused by: ERROR 23505: The statement was aborted because it would have caused a duplicate key value in a unique or prim\ ary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
      at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
      at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown Source)
      at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source)
      at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source)
      at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source)
      at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source)
      at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
      at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)

      when running normal select queries as well - one hits exception, stack trace:

      2008-11-27 01:54:00,216 ERROR metadata.Hive (Hive.java:getTable(275)) - NoSuchObjectException(message:default.dummySrc t\
      able not found)
      at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:347)
      at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:433)
      at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:472)
      at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:272)
      at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:254)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:544)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:3192)
      at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:73)
      at org.apache.hadoop.hive.ql.QTestUtil.analyzeAST(QTestUtil.java:672)
      at org.apache.hadoop.hive.ql.parse.TestParseNegative.testParseNegative_unknown_table1(TestParseNegative.java:231)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at junit.framework.TestCase.runTest(TestCase.java:154)
      at junit.framework.TestCase.runBare(TestCase.java:127)
      at junit.framework.TestResult$1.protect(TestResult.java:106)
      at junit.framework.TestResult.runProtected(TestResult.java:124)
      at junit.framework.TestResult.run(TestResult.java:109)
      at junit.framework.TestCase.run(TestCase.java:118)
      at junit.framework.TestSuite.runTest(TestSuite.java:208)
      at junit.framework.TestSuite.run(TestSuite.java:203)
      at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:297)
      at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:672)
      at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:567)

      Attachments

        1. hive-84.patch
          15 kB
          Prasad Chakka

        Activity

          People

            prasadc Prasad Chakka
            jsensarma Joydeep Sen Sarma
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: