Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9404

NPE in org.apache.hadoop.hive.metastore.txn.TxnHandler.determineDatabaseProduct()

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.15.0
    • Fix Version/s: 1.0.0
    • Component/s: Transactions
    • Labels:
      None

      Description

      Caused by: java.lang.NullPointerException
      
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.determineDatabaseProduct(TxnHandler.java:1015)
      
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkRetryable(TxnHandler.java:906)
      
      	at org.apache.hadoop.hive.metastore.txn.TxnHandler.getOpenTxns(TxnHandler.java:238)
      
      	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_open_txns(HiveMetaStore.java:5321)
      
      
      1. HIVE-9404.patch
        3 kB
        Eugene Koifman

        Issue Links

          Activity

          Hide
          ekoifman Eugene Koifman added a comment -

          Alan Gates could you review please? This is a small followup to HIVE-9390

          Show
          ekoifman Eugene Koifman added a comment - Alan Gates could you review please? This is a small followup to HIVE-9390
          Hide
          hiveqa Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12692930/HIVE-9404.patch

          ERROR: -1 due to 10 failed/errored test(s), 7225 tests executed
          Failed tests:

          TestMiniTezCliDriver-enforce_order.q-cbo_windowing.q-tez_schema_evolution.q-and-12-more - did not produce a TEST-*.xml file
          TestMiniTezCliDriver-update_orig_table.q-vectorization_13.q-update_after_multiple_inserts.q-and-12-more - did not produce a TEST-*.xml file
          TestMiniTezCliDriver-vector_coalesce.q-dynamic_partition_pruning.q-tez_union_group_by.q-and-12-more - did not produce a TEST-*.xml file
          TestMiniTezCliDriver-vector_data_types.q-orc_merge5.q-vectorization_limit.q-and-12-more - did not produce a TEST-*.xml file
          TestMiniTezCliDriver-vector_partitioned_date_time.q-vector_non_string_partition.q-tez_dml.q-and-12-more - did not produce a TEST-*.xml file
          TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more - did not produce a TEST-*.xml file
          TestMinimrCliDriver-leftsemijoin_mr.q-bucket5.q-root_dir_external_table.q-and-1-more - did not produce a TEST-*.xml file
          TestNegativeMinimrCliDriver-udf_local_resource.q-mapreduce_stack_trace_turnoff_hadoop20.q-mapreduce_stack_trace.q-and-5-more - did not produce a TEST-*.xml file
          org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin10
          org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2412/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2412/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2412/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 10 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12692930 - PreCommit-HIVE-TRUNK-Build

          Show
          hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12692930/HIVE-9404.patch ERROR: -1 due to 10 failed/errored test(s), 7225 tests executed Failed tests: TestMiniTezCliDriver-enforce_order.q-cbo_windowing.q-tez_schema_evolution.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-update_orig_table.q-vectorization_13.q-update_after_multiple_inserts.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-vector_coalesce.q-dynamic_partition_pruning.q-tez_union_group_by.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-vector_data_types.q-orc_merge5.q-vectorization_limit.q-and-12-more - did not produce a TEST-*.xml file TestMiniTezCliDriver-vector_partitioned_date_time.q-vector_non_string_partition.q-tez_dml.q-and-12-more - did not produce a TEST-*.xml file TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-leftsemijoin_mr.q-bucket5.q-root_dir_external_table.q-and-1-more - did not produce a TEST-*.xml file TestNegativeMinimrCliDriver-udf_local_resource.q-mapreduce_stack_trace_turnoff_hadoop20.q-mapreduce_stack_trace.q-and-5-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketmapjoin10 org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2412/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2412/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2412/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed This message is automatically generated. ATTACHMENT ID: 12692930 - PreCommit-HIVE-TRUNK-Build
          Hide
          ekoifman Eugene Koifman added a comment -

          these test failures are not related - they've been happening in other runs

          Show
          ekoifman Eugene Koifman added a comment - these test failures are not related - they've been happening in other runs
          Hide
          alangates Alan Gates added a comment -

          +1, the changes look fine.

          As a side note, I'm wondering if in isRetryable the line if("08S01".equalsIgnoreCase(sqlException.getSQLState())) shouldn't include a check whether dbProduct is MySQL (I know I should have said this in the review of HIVE-9390 instead).

          Show
          alangates Alan Gates added a comment - +1, the changes look fine. As a side note, I'm wondering if in isRetryable the line if("08S01".equalsIgnoreCase(sqlException.getSQLState())) shouldn't include a check whether dbProduct is MySQL (I know I should have said this in the review of HIVE-9390 instead).
          Hide
          ekoifman Eugene Koifman added a comment -

          I actually saw this state on Azure which uses MsSQL. More importantly, we often get this when checking out a connection from the pool, i.e. isRetryable() has to handle not having a Connection.... I can add the check but I think it will be of limited use at least for this type of error

          Show
          ekoifman Eugene Koifman added a comment - I actually saw this state on Azure which uses MsSQL. More importantly, we often get this when checking out a connection from the pool, i.e. isRetryable() has to handle not having a Connection.... I can add the check but I think it will be of limited use at least for this type of error
          Hide
          alangates Alan Gates added a comment -

          Ok, that's fine. +1 for the patch then.

          Show
          alangates Alan Gates added a comment - Ok, that's fine. +1 for the patch then.
          Hide
          ekoifman Eugene Koifman added a comment -

          Thanks Alan Gates for the review

          Show
          ekoifman Eugene Koifman added a comment - Thanks Alan Gates for the review
          Hide
          vikram.dixit Vikram Dixit K added a comment -

          Eugene Koifman Does this need to ported to branch 1.0?

          Show
          vikram.dixit Vikram Dixit K added a comment - Eugene Koifman Does this need to ported to branch 1.0?
          Hide
          ekoifman Eugene Koifman added a comment -

          Vikram Dixit K yes but only if you are also including HIVE-9390

          Show
          ekoifman Eugene Koifman added a comment - Vikram Dixit K yes but only if you are also including HIVE-9390
          Hide
          ekoifman Eugene Koifman added a comment -

          Committed to 0.14 and 1.0

          Show
          ekoifman Eugene Koifman added a comment - Committed to 0.14 and 1.0
          Hide
          thejas Thejas M Nair added a comment -

          Updating release version for jiras resolved in 1.0.0 .

          Show
          thejas Thejas M Nair added a comment - Updating release version for jiras resolved in 1.0.0 .
          Hide
          thejas Thejas M Nair added a comment -

          This issue has been fixed in Apache Hive 1.0.0. If there is any issue with the fix, please open a new jira to address it.

          Show
          thejas Thejas M Nair added a comment - This issue has been fixed in Apache Hive 1.0.0. If there is any issue with the fix, please open a new jira to address it.

            People

            • Assignee:
              ekoifman Eugene Koifman
              Reporter:
              ekoifman Eugene Koifman
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development