[PIG-2085] HBaseStorage fails with multiple STORE statements

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Assignee: William W. Graham Jr
• Reporter: William W. Graham Jr
• Created: 20/May/11 21:20
• Updated: 24/Apr/12 15:26
• Resolved: 27/Mar/12 20:50

Description

Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:

raw = LOAD 'hbase_split_load_bug.txt' AS
      (f1: chararray, f2:chararray);

SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');

STORE apples INTO 'hbase://test_table'
   USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');

STORE oranges INTO 'hbase://test_table'
   USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');

The server throws the following exception after apples is successfully stored:

Backend error message
---------------------
java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
        at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
        at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
        at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
        at org.apache.hadoop.mapred.Child.main(Child.java:170)