Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18086

NullPointerException initializing query job when non-native table has partition columns

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None
    • Environment:

      CDH 5.5.5 on RHEL 6.8

      Description

      First, some background info: A non-native table can be created with partition columns defined. However, the existence of partition columns for a non-native table is problematic. Nothing disallows the table creation, and the documentation [1] does not mention that non-native tables cannot have partition columns. In fact, it suggests that "PARTITIONED BY" can be specified.

      When launching the MR job for any query a NullPointerException is thrown. A null path value originates in org.apache.hadoop.hive.ql.exec.Utilities#createDummyFileForEmptyTable(...) because "if this isn't a hive table we can't create an empty file for it" and is added to the list of input paths that the org.apache.hadoop.hive.ql.exec.mr.ExecDriver provides to the FileInputFormat, ultimately causing the NPE in the Path constructor because the child path is null.

      Total jobs = 1
      Launching Job 1 out of 1
      Number of reduce tasks is set to 0 since there's no reduce operator
      java.lang.NullPointerException
      	at org.apache.hadoop.fs.Path.<init>(Path.java:115)
      	at org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:438)
      	at org.apache.hadoop.hive.ql.exec.Utilities.setInputPaths(Utilities.java:3550)
      	at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371)
      	at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
      	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
      	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
      	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1690)
      	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1449)
      	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1235)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1053)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
      	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:209)
      	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
      	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:372)
      	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759)
      	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
      	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:617)
      

      We encountered this bug using the HBaseStorageHandler.

      [1] https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                noslowerdna Andrew Olson
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: