Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3285

Jobs using HBaseStorage fail to ship dependency jars

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.13.0
    • None
    • None

    Description

      Launching a job consuming HBaseStorage fails out of the box. The user must specify -Dpig.additional.jars for HBase and all of its dependencies. Exceptions look something like this:

      2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
      	at org.apache.hadoop.hbase.io.HbaseObjectWritable.<clinit>(HbaseObjectWritable.java:266)
      	at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
      	at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
      	at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
      	at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
      	at $Proxy7.getProtocolVersion(Unknown Source)
      	at org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
      	at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
      

      Attachments

        1. 0001-PIG-3285-Add-HBase-dependency-jars.patch
          5 kB
          Nick Dimiduk
        2. 0001-PIG-3285-Add-HBase-dependency-jars.patch
          4 kB
          Nick Dimiduk
        3. 0001-PIG-3285-Add-HBase-dependency-jars.patch
          5 kB
          Nick Dimiduk
        4. 0001-PIG-3285-Add-HBase-dependency-jars.patch
          4 kB
          Nick Dimiduk
        5. 0001-PIG-3285-Add-HBase-dependency-jars.patch
          2 kB
          Nick Dimiduk
        6. 0001-PIG-3285-Add-HBase-dependency-jars.patch
          2 kB
          Nick Dimiduk
        7. 1.pig
          0.1 kB
          Nick Dimiduk
        8. 1.txt
          0.0 kB
          Nick Dimiduk
        9. 2.pig
          0.1 kB
          Nick Dimiduk

        Issue Links

          Activity

            People

              ndimiduk Nick Dimiduk
              ndimiduk Nick Dimiduk
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: