[PIG-2999] Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11, 0.12.0
• Fix Version/s: 0.11
• Assignee: Koji Noguchi
• Reporter: Koji Noguchi
• Created: 23/Oct/12 03:15
• Updated: 22/Feb/13 04:54
• Resolved: 25/Oct/12 00:14

Description

I think I broke the build from PIG-2975. I see couple of tests failing at BinInterSedesTupleRawComparator.

12/10/22 22:26:15 WARN mapred.LocalJobRunner: job_local_0022
java.nio.BufferUnderflowException
	at java.nio.Buffer.nextGetIndex(Buffer.java:478)
	at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:387)
	at org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinInterSedesDatum(BinInterSedes.java:829)
	at org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinSedesTuple(BinInterSedes.java:732)
	at org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compare(BinInterSedes.java:695)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSecondaryKeyComparator.compare(PigSecondaryKeyComparator.java:78)
	at org.apache.hadoop.mapred.Merger$MergeQueue.lessThan(Merger.java:373)
	at org.apache.hadoop.util.PriorityQueue.downHeap(PriorityQueue.java:139)
	at org.apache.hadoop.util.PriorityQueue.adjustTop(PriorityQueue.java:103)
	at org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:335)
	at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:350)
	at org.apache.hadoop.mapred.ReduceTask$4.next(ReduceTask.java:625)
	at org.apache.hadoop.mapreduce.ReduceContext.nextKeyValue(ReduceContext.java:117)
	at org.apache.hadoop.mapreduce.ReduceContext.nextKey(ReduceContext.java:92)
	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:175)
	at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)

Issue Links

• is duplicated by: PIG-2984 MergeJoin, MergeSparseJoin, and Order e2e test fail
• is related to: PIG-3003 org.apache.pig.test.TestStreamingLocal.testSimpleOrderedReduceSideStreamingAfterFlatten fails with error related to BinInterSedesTupleRawComparator.compare