[CASSANDRA-5867] The Pig CqlStorage/AbstractCassandraStorage classes don't handle collection types

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 1.2.10, 2.0.1
• Labels: pig
• Severity: Normal
• Since Version: 1.2.7
• Assignee: Alex Liu
• Reporter: Jeremy Hanna

Authors

Alex Liu

Reviewers

Brandon Williams

• Created: 09/Aug/13 14:04
• Updated: 16/Apr/19 09:32
• Resolved: 10/Sep/13 18:47

Description

The CqlStorage class gets the Pig data type for values from the AbstractCassandraStorage class, in the getPigType method. If it isn't a known data type, it makes the value into a ByteArray. Currently there aren't any cases there for lists, maps, and sets.
https://github.com/apache/cassandra/blob/cassandra-1.2.8/src/java/org/apache/cassandra/hadoop/pig/AbstractCassandraStorage.java#L336

See this describe output from the grunt shell:

grunt> describe listdata ;                                        
listdata: {id: (name: chararray,value: int),alist: (name: chararray,value: bytearray),amap: (name: chararray,value: bytearray),aset: (name: chararray,value: bytearray)}

where the cql data structures had this schema:

CREATE TABLE alltypes (
  id int PRIMARY KEY,
  alist list<text>,
  amap map<text, text>,
  aset set<text>

It turns out that if you cast the map in grunt to a pig map, then it sort of works, but I don't think we should probably use a pig map. Lists don't appear to work at all, as there is no Pig analogue. I think you could probably just do a UDF to cast these things, but we already have all of the type information, so we just need to change them to tuples or bags or whatever.

Issue Links

• breaks: CASSANDRA-6073 Changes for Pig collections break CQL prepared statements