[CHUKWA-673] pig script error

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.5.0
• Component/s: Scripts

Environment

Hadoop1.03+Hbase0.92.0+Chukwa0.50+Java6

• Assignee: Unassigned
• Reporter: zhang xiaolong
• Created: 05/Dec/12 07:26
• Updated: 06/Dec/12 06:48

Description

Pig script ClusterSummary.pig may have some problems.
"... network:TxErrors network:TxPackets','-loadKey -gt $START -caster Utf8StorageConverter') AS ...." in ClusterSummary.pig didn't piont out "-loadKey (true/false)",and should be "-loadKey true".
And many rows stored in "hbase://SystemMetrics" didn't have value of "tags:cluster". The script use rowkey and "tags:cluster" to generate rowId. This may cause problems. I suggest use rowkey and "XX:ctags" to generate rowId.

-------------------------------------------------------------------------------------
When I run Pig script ClusterSummary.pig , got error:
Unable to recreate exception from backed error: java.io.IOException: java.lang.IllegalArgumentException: Row key is invalid
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:465)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:428)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:408)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:262)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.IllegalArgumentException: Row key is invalid
at org.apache.hadoop.hbase.client.Put.<init>(Put.java:93)
at org.apache.hadoop.hbase.client.Put.<init>(Put.java:72)
at org.apache.hadoop.hbase.client.Put.<init>(Put.java:63)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.createPut(HBaseStorage.java:712)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.putNext(HBaseStorage.java:660)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:587)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:463)
... 11 more
-------------------------------------------------------------------------------------