Description
Table is not partitioned and has an RCFile for the data.
The following pig script will cause the MR jobs to fail:
a = load 'default.table' USING org.apache.hcatalog.pig.HCatLoader();
b = order a by id;
dump b;
If I prefix the order by with a foreach statement, then the job will pass.
The MR job fails with the following exception:
Error: java.lang.ClassNotFoundException: javax.jdo.JDOException at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:346) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:333) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:371) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:278) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:248) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:114) at org.apache.hcatalog.mapreduce.InitializeInput.createHiveMetaClient(InitializeInput.java:58) at org.apache.hcatalog.mapreduce.InitializeInput.getSerializedHcatKeyJobInfo(InitializeInput.java:85) at org.apache.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:73) at org.apache.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:40) at org.apache.hcatalog.pig.HCatLoader.setLocation(HCatLoader.java:116) at org.apache.pig.impl.builtin.SampleLoader.setLocation(SampleLoader.java:98) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.mergeSplitSpecificConf(PigInputFormat.java:134) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputFormat.java:112) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:489) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093) at org.apache.hadoop.mapred.Child.main(Child.java:249)
Attachments
Attachments
Issue Links
- is related to
-
PIG-2666 LoadFunc.setLocation() is not called when pig script only has Order By
- Closed