Hive
  1. Hive
  2. HIVE-6953

All CompactorTest failing with Table/View 'NEXT_TXN_ID' does not exist

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.13.0
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None

      Description

      When I'm running all tests through the command 'mvn clean install -Phadoop-1', all CompactorTest classes TestInitiator, TestWorker, TestCleaner fail with following exception :

      org.apache.hadoop.hive.metastore.api.MetaException: Unable to select from transaction database java.sql.SQLSyntaxErrorException: Table/View 'NEXT_TXN_ID' does not exist.
              at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
              at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
              at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
              at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
      
      ....
      Caused by: java.sql.SQLException: Table/View 'NEXT_TXN_ID' does not exist.
              at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
              at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      
      

      This is happening on branch-0.13. Has anyone faced this problem?

      Owen O'Malley or someone else help me solve this. Do i have to set anything?

      1. TEST-org.apache.hadoop.hive.ql.txn.compactor.TestWorker.xml
        113 kB
        Jaideep Dhok
      2. TEST-org.apache.hadoop.hive.ql.txn.compactor.TestInitiator.xml
        171 kB
        Amareshwari Sriramadasu
      3. TEST-org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.xml
        74 kB
        Jaideep Dhok
      4. nohup.out.gz
        19 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          Hide
          Amareshwari Sriramadasu added a comment -

          The test failures were because of HIVE-6877. After applying the patch from HIVE-6877 into branch-0.13, all the tests are passing

          Show
          Amareshwari Sriramadasu added a comment - The test failures were because of HIVE-6877 . After applying the patch from HIVE-6877 into branch-0.13, all the tests are passing
          Hide
          Amareshwari Sriramadasu added a comment - - edited

          Also, there are some tests failing randomly because they fail to create path in /user/hive/warehouse

          For ex: org.apache.hadoop.hive.ql.io.TestSymlinkTextInputFormat failed with following errors

          log4j:ERROR Could not read configuration file from URL [file:/<hive-path>/ql/target/tmp/conf/hive-log4j.properties].
          java.io.FileNotFoundException: /<hive-path>/ql/target/tmp/conf/hive-log4j.properties (No such file or directory)
          ....
          
          FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:file:/user/hive/warehouse/text_symlink_text is not a directory or unable to create one)
          

          So, most probably something is getting cleaned-up. But I could not find where to start.

          Can someone help me to find the root cause? Where can i start to look at it.

          Show
          Amareshwari Sriramadasu added a comment - - edited Also, there are some tests failing randomly because they fail to create path in /user/hive/warehouse For ex: org.apache.hadoop.hive.ql.io.TestSymlinkTextInputFormat failed with following errors log4j:ERROR Could not read configuration file from URL [file:/<hive-path>/ql/target/tmp/conf/hive-log4j.properties]. java.io.FileNotFoundException: /<hive-path>/ql/target/tmp/conf/hive-log4j.properties (No such file or directory) .... FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:file:/user/hive/warehouse/text_symlink_text is not a directory or unable to create one) So, most probably something is getting cleaned-up. But I could not find where to start. Can someone help me to find the root cause? Where can i start to look at it.
          Hide
          Jaideep Dhok added a comment -

          Tests are failing for me as well. I did a fresh checkout of the git repo, and ran the test with hadoop-1 profile.

          Show
          Jaideep Dhok added a comment - Tests are failing for me as well. I did a fresh checkout of the git repo, and ran the test with hadoop-1 profile.
          Hide
          Amareshwari Sriramadasu added a comment -

          Attaching TestInitiator.xml

          Show
          Amareshwari Sriramadasu added a comment - Attaching TestInitiator.xml
          Hide
          Amareshwari Sriramadasu added a comment -

          The machine on which I'm running is a Linux machine. Same thing happens on Mac as well.

          uname -a
          Linux <hostname> 2.6.32-21-server #32-Ubuntu SMP Fri Apr 16 09:17:34 UTC 2010 x86_64 GNU/Linux
          
          $ java -version
          java version "1.6.0_26"
          Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
          Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
          
          Show
          Amareshwari Sriramadasu added a comment - The machine on which I'm running is a Linux machine. Same thing happens on Mac as well. uname -a Linux <hostname> 2.6.32-21-server #32-Ubuntu SMP Fri Apr 16 09:17:34 UTC 2010 x86_64 GNU/Linux $ java -version java version "1.6.0_26" Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
          Hide
          Amareshwari Sriramadasu added a comment -

          The nohup test output

          Show
          Amareshwari Sriramadasu added a comment - The nohup test output
          Hide
          Amareshwari Sriramadasu added a comment -

          Thanks Alan Gates and Harish Butani for trying. The tests are passing when i run them individually. When all the tests are run together, they are failing.

          Here is what i have done :

          git clone https://github.com/apache/hive apache-hive
          git checkout branch-0.13
          nohup mvn clean install -Phadoop-1 &
          

          Attaching nohup output for reference.

          Is there anything in your logs indicating it tried to create the tables and failed?

          Will check and update.

          Are you doing anything in your build to turn off the hive.in.test config value?

          No.

          I'm thinking the test db or conf is getting cleaned up by some-other means, when all the tests are run together.

          Show
          Amareshwari Sriramadasu added a comment - Thanks Alan Gates and Harish Butani for trying. The tests are passing when i run them individually. When all the tests are run together, they are failing. Here is what i have done : git clone https://github.com/apache/hive apache-hive git checkout branch-0.13 nohup mvn clean install -Phadoop-1 & Attaching nohup output for reference. Is there anything in your logs indicating it tried to create the tables and failed? Will check and update. Are you doing anything in your build to turn off the hive.in.test config value? No. I'm thinking the test db or conf is getting cleaned up by some-other means, when all the tests are run together.
          Hide
          Harish Butani added a comment -

          Just ran the tests you mention: TestInitiator, TestWorker, TestCleaner on 0.13
          I cannot reproduce the errors either.

          Show
          Harish Butani added a comment - Just ran the tests you mention: TestInitiator, TestWorker, TestCleaner on 0.13 I cannot reproduce the errors either.
          Hide
          Alan Gates added a comment -

          Amareshwari Sriramadasu, I can't reproduce this. Tests pass for me in 0.13 and trunk. These tests should be creating the tables they need as part of the tests (see TxnHandler.checkQFileTestHack()). Is there anything in your logs indicating it tried to create the tables and failed? Are you doing anything in your build to turn off the hive.in.test config value? This would prevent the tables being created.

          Show
          Alan Gates added a comment - Amareshwari Sriramadasu , I can't reproduce this. Tests pass for me in 0.13 and trunk. These tests should be creating the tables they need as part of the tests (see TxnHandler.checkQFileTestHack()). Is there anything in your logs indicating it tried to create the tables and failed? Are you doing anything in your build to turn off the hive.in.test config value? This would prevent the tables being created.
          Hide
          Amareshwari Sriramadasu added a comment -

          There are no failures in trunk, all tests pass. Harish Butani, do you think anything is missing in branch-0.13? Looking at commits, i couldnt figure out.

          Show
          Amareshwari Sriramadasu added a comment - There are no failures in trunk, all tests pass. Harish Butani , do you think anything is missing in branch-0.13? Looking at commits, i couldnt figure out.

            People

            • Assignee:
              Alan Gates
              Reporter:
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development