[HIVE-5431] PassthroughOutputFormat SH changes causes IllegalArgumentException

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.12.0
• Fix Version/s: 0.12.0
• Assignee: Sushanth Sowmyan
• Reporter: Sushanth Sowmyan
• Created: 03/Oct/13 03:35
• Updated: 15/Oct/13 23:29
• Resolved: 05/Oct/13 16:18

Description

The recent changes with HIVE-4331 introduced a new key "hive.passthrough.storagehandler.of", whose value is set only on storage handler writes, but obviously, will not be set on reads. However, PlanUtils.configureJobPropertiesForStorageHandler winds up trying to set the key for both cases into jobProperties, which cause any reads that are not preceeded by writes to fail.

Basically, if you have a .q in which you insert data into a hbase table and then read it, it's okay. If you have a .q in which you only read data, it throws an IllegalArgumentException, like so:

2013-09-30 16:20:01,989 ERROR CliDriver (SessionState.java:printError(419)) - Failed with exception java.io.IOException:java.lang.IllegalArgumentException: Property value must not be null
java.io.IOException: java.lang.IllegalArgumentException: Property value must not be null
        at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:551)
        at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:489)
        at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
        at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1471)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
        at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:446)
        at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:456)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:737)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.IllegalArgumentException: Property value must not be null
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
        at org.apache.hadoop.conf.Configuration.set(Configuration.java:810)
        at org.apache.hadoop.conf.Configuration.set(Configuration.java:792)
        at org.apache.hadoop.hive.ql.exec.Utilities.copyTableJobPropertiesToConf(Utilities.java:1826)
        at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:380)
        at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:515)
        ... 17 more

Issue Links

• is related to: HIVE-5515 Writing to an HBase table throws IllegalArgumentException, failing job submission
• relates to: HIVE-4331 Integrated StorageHandler for Hive and HCat using the HiveStorageHandler