[CRUNCH-147] Pipeline.writeTextFile should work with Avro data

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.5.0
• Component/s: Core
• Assignee: Josh Wills
• Reporter: Dave Beech
• Created: 22/Jan/13 14:21
• Updated: 22/Feb/13 06:57
• Resolved: 23/Jan/13 16:17

Description

I'd find it useful to be able to dump Avro collections to a text file (as json) during development to have a quick look inside without having to mess around with avro-tools. The current Pipeline.writeTextFile method expects Writables and gives the following error when you attempt to pass Avro to it:

Exception in thread "main" java.lang.ClassCastException: class my.avro.Record
at java.lang.Class.asSubclass(Class.java:3018)
at org.apache.crunch.types.writable.Writables.records(Writables.java:250)
at org.apache.crunch.types.writable.WritableTypeFamily.records(WritableTypeFamily.java:86)
at org.apache.crunch.types.PTypeUtils.convert(PTypeUtils.java:61)
at org.apache.crunch.types.writable.WritableTypeFamily.as(WritableTypeFamily.java:135)

Issue Links

• relates to: CRUNCH-52 Writing to a text file with avro types fills a file with the tostring of AvroWrapper