Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: 1.0.9, 1.1.0
    • Component/s: None
    • Labels:
      None

      Description

      It appears that some changes need to be made to support CompositeColumns. Right now if you try to load and use a column family that utilizes composite columns you get the following exception[1].

      It appears to me that we need to modify the storage handler for Pig to support this scenario.

      [1]

      ================================================================================
      Backend error message
      ---------------------
      java.lang.RuntimeException: Unexpected data type -1 found in stream.
      at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478)
      at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
      at org.apache.pig.data.BinInterSedes.writeBag(BinInterSedes.java:522)
      at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:361)
      at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
      at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
      at org.apache.pig.data.BinSedesTuple.write(BinSedesTuple.java:57)
      at org.apache.pig.impl.io.PigNullableWritable.write(PigNullableWritable.java:123)
      at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90)
      at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77)
      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1061)
      at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
      at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:116)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:239)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:272)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
      at org.apache.hadoop.mapred.Child.main(Child.java:266)

      Backend error message
      ---------------------
      java.lang.Throwable: Child Error
      at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
      Caused by: java.io.IOException: Task process exit with nonzero status of 65.
      at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)

        Attachments

        1. 3684-jalkanen.txt
          6 kB
          Janne Jalkanen
        2. 3684-jalkanen-test.txt
          1 kB
          Janne Jalkanen
        3. 3684-jalkanen-test-v2.txt
          2 kB
          Janne Jalkanen

          Activity

            People

            • Assignee:
              jalkanen Janne Jalkanen
              Reporter:
              bcoverston Benjamin Coverston
              Reviewer:
              Brandon Williams
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: