Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2656

HBaseStorage, Job setup fails

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: 0.9.2
    • Fix Version/s: None
    • Component/s: piggybank
    • Environment:

      Hadoop 1.x, Pig 0.9.2, Ubuntu 64bit.

      Description

        1. I'm trying to run the following Pig Script:

      /* ## Set the environment ## */
      SET mapred.job.queue.name Pig;
      SET mapred.output.compress true;
      SET avro.output.codec snappy;
      SET hbase.zookeeper.quorum 'somehost.org';
      SET hbase.zookeeper.property.clientPort 2222;
      SET hbase.cluster.distributed true;

      /* ## Read our paths ## */
      A = LOAD 'hdfs://somehost.org:9000/Pig/Input.avro'
      USING org.apache.pig.piggybank.storage.avro.AvroStorage ('same', 'hdfs://somehost.org:9000/Pig/Input.avro');

      B = FOREACH A GENERATE objUUID, objTimestamp, objGame;

      /* ## Write out results ## */
      STORE B INTO 'hbase://Exported'
      USING org.apache.pig.backend.hadoop.hbase.HBaseStorage ('CF:objUUID CF:objTimestamp CF:objGame');

        1. I get the following outputs from the tasks:

      java.lang.RuntimeException: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@12789d2 closed
      at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:200)
      at org.apache.pig.backend.hadoop.hbase.HBaseStorage.getOutputFormat(HBaseStorage.java:521)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:91)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.<init>(PigOutputCommitter.java:69)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:279)
      at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:416)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
      at org.apache.hadoop.mapred.Child.main(Child.java:249)
      Caused by: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@12789d2 closed
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:794)
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782)
      at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:249)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:213)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
      at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:195)
      ... 11 more

      java.lang.RuntimeException: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@20f237 closed
      at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:200)
      at org.apache.pig.backend.hadoop.hbase.HBaseStorage.getOutputFormat(HBaseStorage.java:521)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:91)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.<init>(PigOutputCommitter.java:69)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:279)
      at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:416)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
      at org.apache.hadoop.mapred.Child.main(Child.java:249)
      Caused by: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@20f237 closed
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:794)
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782)
      at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:249)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:213)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
      at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:195)
      ... 11 more

      java.lang.RuntimeException: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1192059 closed
      at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:200)
      at org.apache.pig.backend.hadoop.hbase.HBaseStorage.getOutputFormat(HBaseStorage.java:521)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:91)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.<init>(PigOutputCommitter.java:69)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:279)
      at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:416)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
      at org.apache.hadoop.mapred.Child.main(Child.java:249)
      Caused by: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@1192059 closed
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:794)
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782)
      at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:249)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:213)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
      at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:195)
      ... 11 more

      java.lang.RuntimeException: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2803d5 closed
      at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:200)
      at org.apache.pig.backend.hadoop.hbase.HBaseStorage.getOutputFormat(HBaseStorage.java:521)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.getCommitters(PigOutputCommitter.java:91)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.<init>(PigOutputCommitter.java:69)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getOutputCommitter(PigOutputFormat.java:279)
      at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:416)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
      at org.apache.hadoop.mapred.Child.main(Child.java:249)
      Caused by: java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@2803d5 closed
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:794)
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:782)
      at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:249)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:213)
      at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
      at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:195)
      ... 11 more

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              catalinalexandru.zamfir@gameloft.com Catalin Alexandru Zamfir
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: