[TEZ-863] NullPointerException in shuffle phase

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.3.0
• Fix Version/s: 0.3.0

Environment

Hadoop 2.3.0, Hive 0.13, Tez 0.3.0

• Assignee: Rajesh Balamohan
• Reporter: Rajesh Balamohan
• Created: 20/Feb/14 11:01
• Updated: 01/Mar/14 04:19
• Resolved: 21/Feb/14 05:41

Description

When running amplab-benchmark (query 2a, 2b, 2c), randomly the following exception is thrown. It is not very consistent though.

Amplab query reference: https://amplab.cs.berkeley.edu/benchmark/
>>>
SELECT SUBSTR(sourceIP, 1, 10), SUM(adRevenue) FROM uservisits GROUP BY SUBSTR(sourceIP, 1, 10)
>>>

Vertex re-running, vertexName=Map 1, vertexId=vertex_1392875986731_0052_2_01
Vertex failed, vertexName=Reducer 2, vertexId=vertex_1392875986731_0052_2_00, diagnostics=[Task failed, taskId=task_1392875986731_0052_2_00_000030, diagnostics=[AttemptID:attempt_1392875986731_0052_2_00_000030_0 Info:Error: java.lang.NullPointerException
at org.apache.tez.runtime.library.common.shuffle.impl.Shuffle.handleEvents(Shuffle.java:175)
at org.apache.tez.runtime.library.input.ShuffledMergedInput.handleEvents(ShuffledMergedInput.java:195)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.handleEvent(LogicalIOProcessorRuntimeTask.java:566)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.access$1100(LogicalIOProcessorRuntimeTask.java:81)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask$1.run(LogicalIOProcessorRuntimeTask.java:628)
at java.lang.Thread.run(Thread.java:744)