Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • Metastore
    • None
    • Reviewed

    Description

      Stack:

      2017-06-30T02:39:43,739 ERROR [HiveServer2-Background-Pool: Thread-2873]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(200)) - MetaException(message:java.lang.NullPointerException)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:6066)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_core(HiveMetaStore.java:3993)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_with_environment_context(HiveMetaStore.java:3944)
              at sun.reflect.GeneratedMethodAccessor142.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
              at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
              at com.sun.proxy.$Proxy32.alter_table_with_environment_context(Unknown Source)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_table_with_environmentContext(HiveMetaStoreClient.java:397)
              at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.alter_table_with_environmentContext(SessionHiveMetaStoreClient.java:325)
              at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:173)
              at com.sun.proxy.$Proxy33.alter_table_with_environmentContext(Unknown Source)
              at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2306)
              at com.sun.proxy.$Proxy33.alter_table_with_environmentContext(Unknown Source)
              at org.apache.hadoop.hive.ql.metadata.Hive.alterTable(Hive.java:624)
              at org.apache.hadoop.hive.ql.exec.DDLTask.alterTable(DDLTask.java:3490)
              at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:383)
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
              at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1905)
              at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1607)
              at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1354)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1123)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1116)
              at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242)
              at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
              at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:334)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:422)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
              at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:348)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.NullPointerException
              at org.apache.hadoop.hive.metastore.cache.SharedCache.getCachedTableColStats(SharedCache.java:140)
              at org.apache.hadoop.hive.metastore.cache.CachedStore.getTableColumnStatistics(CachedStore.java:1409)
              at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
              at com.sun.proxy.$Proxy28.getTableColumnStatistics(Unknown Source)
              at org.apache.hadoop.hive.metastore.HiveAlterHandler.alterTableUpdateTableColumnStats(HiveAlterHandler.java:800)
              at org.apache.hadoop.hive.metastore.HiveAlterHandler.alterTable(HiveAlterHandler.java:257)
              at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_core(HiveMetaStore.java:3973)
              ... 43 more
      

      Attachments

        1. HIVE-17007.1.patch
          1 kB
          Daniel Dai

        Issue Links

          Activity

            daijy Daniel Dai added a comment -

            It should be a simple NPE check as null result is common and will be handled by HiveAlterHandler.

            daijy Daniel Dai added a comment - It should be a simple NPE check as null result is common and will be handled by HiveAlterHandler.
            hiveqa Hive QA added a comment -

            Here are the results of testing the latest attachment:
            https://issues.apache.org/jira/secure/attachment/12875338/HIVE-17007.1.patch

            ERROR: -1 due to no test(s) being added or modified.

            ERROR: -1 due to 10 failed/errored test(s), 10830 tests executed
            Failed tests:

            org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=143)
            org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=140)
            org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=157)
            org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=145)
            org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=99)
            org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=232)
            org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=232)
            org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=177)
            org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=177)
            org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=177)
            

            Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5858/testReport
            Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5858/console
            Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5858/

            Messages:

            Executing org.apache.hive.ptest.execution.TestCheckPhase
            Executing org.apache.hive.ptest.execution.PrepPhase
            Executing org.apache.hive.ptest.execution.ExecutionPhase
            Executing org.apache.hive.ptest.execution.ReportingPhase
            Tests exited with: TestsFailedException: 10 tests failed
            

            This message is automatically generated.

            ATTACHMENT ID: 12875338 - PreCommit-HIVE-Build

            hiveqa Hive QA added a comment - Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12875338/HIVE-17007.1.patch ERROR: -1 due to no test(s) being added or modified. ERROR: -1 due to 10 failed/errored test(s), 10830 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=143) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=140) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=145) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=99) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=232) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=232) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=177) org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema (batchId=177) org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=177) Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5858/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5858/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5858/ Messages: Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed This message is automatically generated. ATTACHMENT ID: 12875338 - PreCommit-HIVE-Build
            daijy Daniel Dai added a comment -

            thejasvgumashta, can you give a review?

            daijy Daniel Dai added a comment - thejas vgumashta , can you give a review?

            +1

            sushanth Sushanth Sowmyan added a comment - +1
            daijy Daniel Dai added a comment -

            Patch pushed to master. Thanks Sushanth for review!

            daijy Daniel Dai added a comment - Patch pushed to master. Thanks Sushanth for review!
            vgarg Vineet Garg added a comment -

            Hive 3.0.0 has been released so closing this jira.

            vgarg Vineet Garg added a comment - Hive 3.0.0 has been released so closing this jira.

            People

              daijy Daniel Dai
              daijy Daniel Dai
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: