[TEZ-2377] RandomWriter ends up using TextOutputFormat instead of SequenceFileOutputFormat

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: 0.7.0
• Target Version/s: 0.7.0
• Assignee: Rajesh Balamohan
• Reporter: Rajesh Balamohan
• Created: 28/Apr/15 09:27
• Updated: 29/Apr/15 23:49
• Resolved: 29/Apr/15 23:49

Description

yarn jar ./dist/tez/tez-tests-0.7.0-SNAPSHOT.jar randomwriter "-Dmapreduce.randomwriter.totalbytes=10737418" /tmp/test1

This ends up generating TextOutputFormat.

yarn jar ./dist/tez/tez-tests-0.7.0-SNAPSHOT.jar sort  
"-Dmapreduce.framework.name=yarn-tez" -r 5 /tmp/test1 /tmp/test_sorted

This ends up throwing error

Failure while running task:java.io.IOException: hdfs://tez-vm:56565/tmp/test1/part-00000 not a SequenceFile
        at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1851)
        at org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1811)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1760)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1774)
        at org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.initialize(SequenceFileRecordReader.java:54)
        at org.apache.tez.mapreduce.lib.MRReaderMapReduce.setupNewRecordReader(MRReaderMapReduce.java:149)
        at org.apache.tez.mapreduce.lib.MRReaderMapReduce.<init>(MRReaderMapReduce.java:78)
        at org.apache.tez.mapreduce.input.MRInput.initializeInternal(MRInput.java:475)
        at org.apache.tez.mapreduce.input.MRInputLegacy.init(MRInputLegacy.java:108)