Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-3684

Composite Column Support for PIG

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 1.0.9, 1.1.0
    • None
    • None
    • Normal

    Description

      It appears that some changes need to be made to support CompositeColumns. Right now if you try to load and use a column family that utilizes composite columns you get the following exception[1].

      It appears to me that we need to modify the storage handler for Pig to support this scenario.

      [1]

      ================================================================================
      Backend error message
      ---------------------
      java.lang.RuntimeException: Unexpected data type -1 found in stream.
      at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478)
      at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
      at org.apache.pig.data.BinInterSedes.writeBag(BinInterSedes.java:522)
      at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:361)
      at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
      at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
      at org.apache.pig.data.BinSedesTuple.write(BinSedesTuple.java:57)
      at org.apache.pig.impl.io.PigNullableWritable.write(PigNullableWritable.java:123)
      at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90)
      at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77)
      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1061)
      at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
      at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:116)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:239)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:232)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:272)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
      at org.apache.hadoop.mapred.Child.main(Child.java:266)

      Backend error message
      ---------------------
      java.lang.Throwable: Child Error
      at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
      Caused by: java.io.IOException: Task process exit with nonzero status of 65.
      at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)

      Attachments

        1. 3684-jalkanen.txt
          6 kB
          Janne Jalkanen
        2. 3684-jalkanen-test.txt
          1 kB
          Janne Jalkanen
        3. 3684-jalkanen-test-v2.txt
          2 kB
          Janne Jalkanen

        Activity

          People

            jalkanen Janne Jalkanen
            bcoverston Benjamin Coverston
            Janne Jalkanen
            Brandon Williams
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: