Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Not A Problem
-
None
-
None
-
None
-
None
Description
Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
raw = LOAD 'hbase_split_load_bug.txt' AS (f1: chararray, f2:chararray); SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange'); STORE apples INTO 'hbase://test_table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple'); STORE oranges INTO 'hbase://test_table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
The server throws the following exception after apples is successfully stored:
Backend error message --------------------- java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233) at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819) at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.Child.main(Child.java:170)