Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6532

NullPointerException in HiveContextAwareRecordReader.initIOContext() when executing Hive query

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: Impala 3.0, Impala 2.12.0
    • Fix Version/s: None
    • Component/s: None

      Description

      https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/1273/console

      02:48:03 [gw15] FAILED metadata/test_partition_metadata.py::TestPartitionMetadata::test_partition_metadata_compatibility[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block] 
      03:29:11 =================================== FAILURES ===================================
      03:29:11  TestPartitionMetadata.test_partition_metadata_compatibility[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block] 
      03:29:11 [gw15] linux2 -- Python 2.7.12 /home/ubuntu/Impala/bin/../infra/python/env/bin/python
      03:29:11 metadata/test_partition_metadata.py:127: in test_partition_metadata_compatibility
      03:29:11     "insert into %s partition (x) values(1,1)" % FQ_TBL_HIVE)
      03:29:11 common/impala_test_suite.py:684: in run_stmt_in_hive
      03:29:11     raise RuntimeError(stderr)
      03:29:11 E   RuntimeError: SLF4J: Class path contains multiple SLF4J bindings.
      03:29:11 E   SLF4J: Found binding in [jar:file:/home/ubuntu/Impala/toolchain/cdh_components/hbase-1.2.0-cdh5.15.0-SNAPSHOT/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      03:29:11 E   SLF4J: Found binding in [jar:file:/home/ubuntu/Impala/toolchain/cdh_components/hadoop-2.6.0-cdh5.15.0-SNAPSHOT/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      03:29:11 E   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      03:29:11 E   SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
      03:29:11 E   scan complete in 3ms
      03:29:11 E   Connecting to jdbc:hive2://localhost:11050
      03:29:11 E   Connected to: Apache Hive (version 1.1.0-cdh5.15.0-SNAPSHOT)
      03:29:11 E   Driver: Hive JDBC (version 1.1.0-cdh5.15.0-SNAPSHOT)
      03:29:11 E   Transaction isolation: TRANSACTION_REPEATABLE_READ
      03:29:11 E   No rows affected (0.045 seconds)
      03:29:11 E   INFO  : Compiling command(queryId=ubuntu_20180215024848_d80982c5-a75c-4441-ab43-68d238eb69ba): insert into test_partition_metadata_compatibility_b2ac5e.part_parquet_tbl_hive partition (x) values(1,1)
      03:29:11 E   INFO  : Semantic Analysis Completed
      03:29:11 E   INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_col0, type:int, comment:null), FieldSchema(name:_col1, type:string, comment:null)], properties:null)
      03:29:11 E   INFO  : Completed compiling command(queryId=ubuntu_20180215024848_d80982c5-a75c-4441-ab43-68d238eb69ba); Time taken: 0.123 seconds
      03:29:11 E   INFO  : Executing command(queryId=ubuntu_20180215024848_d80982c5-a75c-4441-ab43-68d238eb69ba): insert into test_partition_metadata_compatibility_b2ac5e.part_parquet_tbl_hive partition (x) values(1,1)
      03:29:11 E   INFO  : Query ID = ubuntu_20180215024848_d80982c5-a75c-4441-ab43-68d238eb69ba
      03:29:11 E   INFO  : Total jobs = 3
      03:29:11 E   INFO  : Launching Job 1 out of 3
      03:29:11 E   INFO  : Starting task [Stage-1:MAPRED] in serial mode
      03:29:11 E   INFO  : Number of reduce tasks is set to 0 since there's no reduce operator
      03:29:11 E   INFO  : number of splits:1
      03:29:11 E   INFO  : Submitting tokens for job: job_local1716894275_0002
      03:29:11 E   INFO  : The url to track the job: http://localhost:8080/
      03:29:11 E   INFO  : Job running in-process (local Hadoop)
      03:29:11 E   INFO  : 2018-02-15 02:48:03,500 Stage-1 map = 0%,  reduce = 0%
      03:29:11 E   ERROR : Ended Job = job_local1716894275_0002 with errors
      03:29:11 E   ERROR : FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
      03:29:11 E   INFO  : MapReduce Jobs Launched: 
      03:29:11 E   INFO  : Stage-Stage-1:  HDFS Read: 0 HDFS Write: 0 FAIL
      03:29:11 E   INFO  : Total MapReduce CPU Time Spent: 0 msec
      03:29:11 E   INFO  : Completed executing command(queryId=ubuntu_20180215024848_d80982c5-a75c-4441-ab43-68d238eb69ba); Time taken: 1.902 seconds
      03:29:11 E   Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2)
      03:29:11 E   Closing: 0: jdbc:hive2://localhost:11050
      03:29:11 ---------------------------- Captured stderr setup -----------------------------
      03:29:11 -- connecting to: localhost:21000
      03:29:11 SET sync_ddl=False;
      03:29:11 -- executing against localhost:21000
      03:29:11 DROP DATABASE IF EXISTS `test_partition_metadata_compatibility_b2ac5e` CASCADE;
      03:29:11 
      03:29:11 SET sync_ddl=False;
      03:29:11 -- executing against localhost:21000
      03:29:11 CREATE DATABASE `test_partition_metadata_compatibility_b2ac5e`;
      03:29:11 
      03:29:11 MainThread: Created database "test_partition_metadata_compatibility_b2ac5e" for test ID "metadata/test_partition_metadata.py::TestPartitionMetadata::()::test_partition_metadata_compatibility[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: avro/snap/block]"

      Hive log for the mapReduce job that is failing:

      2018-02-15 02:48:02,626 WARN mapred.LocalJobRunner (LocalJobRunner.java:run(587)) - job_local1716894275_0002
      java.lang.Exception: java.io.IOException: java.lang.reflect.InvocationTargetException
      at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549)
      Caused by: java.io.IOException: java.lang.reflect.InvocationTargetException
      at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
      at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
      at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:267)
      at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:213)
      at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:334)
      at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:695)
      at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:438)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
      at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.reflect.InvocationTargetException
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:253)
      ... 12 more
      Caused by: java.lang.NullPointerException
      at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.initIOContext(HiveContextAwareRecordReader.java:171)
      at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.initIOContext(HiveContextAwareRecordReader.java:208)
      at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:70)
      ... 17 more

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                joemcdonnell Joe McDonnell
                Reporter:
                bikramjeet.vig Bikramjeet Vig
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: