[TEZ-616] OrderedWordCount with -generateSplitsInClient true throws NPE.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.2.0
• Fix Version/s: 0.2.0
• Assignee: Hitesh Shah
• Reporter: Tassapol Athiapinya
• Created: 14/Nov/13 17:28
• Updated: 01/Dec/13 20:21
• Resolved: 15/Nov/13 15:36

Description

OrderedWordCount with -generateSplitsInClient true returns NPE.

2013-11-14 05:05:56,229||INFO|RUNNING:
/usr/lib/hadoop/bin/hadoop jar
/usr/lib/tez/lib/tez-mapreduce-examples.jar
orderedwordcount tezSimpleSplitClientInput tezSimpleSplitClientInputOutput
-generateSplitsInClient true
2013-11-14 05:05:57,230||INFO|13/11/14 05:05:57 INFO
client.RMProxy: Connecting to ResourceManager

2013-11-14 05:05:57,901||INFO|13/11/14 05:05:57 INFO
examples.OrderedWordCount: Creating Tez Session
2013-11-14 05:05:57,933||INFO|13/11/14 05:05:57 INFO
client.RMProxy: Connecting to ResourceManager 
2013-11-14 05:05:58,391||INFO|13/11/14 05:05:58 INFO
impl.YarnClientImpl: Submitted application application_1384284330310_0018
to ResourceManager 
2013-11-14 05:05:58,391||INFO|13/11/14 05:05:58 INFO
examples.OrderedWordCount: Created Tez Session
2013-11-14 05:05:58,393||INFO|13/11/14 05:05:58 INFO
examples.OrderedWordCount: Running OrderedWordCount DAG, dagIndex=1,
inputPath=tezSimpleSplitClientInput,
outputPath=tezSimpleSplitClientInputOutput
2013-11-14 05:05:58,399||INFO|13/11/14 05:05:58 INFO
hadoop.MRHelpers: Generating new input splits,
splitsDir=hdfs://<host>:8020/user/hrt_qa/.staging/appl
ication_1384284330310_0018
2013-11-14 05:05:58,426||INFO|13/11/14 05:05:58 INFO
input.FileInputFormat: Total input paths to process : 5
2013-11-14 05:05:58,438||INFO|13/11/14 05:05:58 INFO
lzo.GPLNativeCodeLoader: Loaded native gpl library
2013-11-14 05:05:58,440||INFO|13/11/14 05:05:58 INFO
lzo.LzoCodec: Successfully loaded & initialized native-lzo library
[hadoop-lzo rev cf4e7cbf8ed0f0622504d008101c2729dc0c9ff3]
2013-11-14 05:05:58,631||INFO|13/11/14 05:05:58 INFO
client.TezSession: Shutting down Tez Session,
sessionName=OrderedWordCountSession,
applicationId=application_1384284330310_0018
2013-11-14 05:05:58,633||INFO|13/11/14 05:05:58 INFO
client.TezSession: Could not connect to AM, killing session via YARN,
sessionName=OrderedWordCountSession,
applicationId=application_1384284330310_0018
2013-11-14 05:05:58,633||INFO|13/11/14 05:05:58 INFO
impl.YarnClientImpl: Killing application application_1384284330310_0018
2013-11-14 05:05:58,641||INFO|java.lang.NullPointerException
2013-11-14 05:05:58,642||INFO|at
java.util.HashMap.putAll(HashMap.java:597)
2013-11-14 05:05:58,642||INFO|at
org.apache.tez.mapreduce.examples.OrderedWordCount.createDAG(OrderedWordCo
unt.java:220)
2013-11-14 05:05:58,643||INFO|at
org.apache.tez.mapreduce.examples.OrderedWordCount.main(OrderedWordCount.j
ava:387)
2013-11-14 05:05:58,643||INFO|at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2013-11-14 05:05:58,643||INFO|at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
57)
2013-11-14 05:05:58,644||INFO|at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
pl.java:43)
2013-11-14 05:05:58,644||INFO|at
java.lang.reflect.Method.invoke(Method.java:601)
2013-11-14 05:05:58,645||INFO|at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriv
er.java:72)
2013-11-14 05:05:58,645||INFO|at
org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
2013-11-14 05:05:58,645||INFO|at
org.apache.tez.mapreduce.examples.ExampleDriver.main(ExampleDriver.java:88
)
2013-11-14 05:05:58,646||INFO|at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2013-11-14 05:05:58,646||INFO|at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
57)
2013-11-14 05:05:58,647||INFO|at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
pl.java:43)
2013-11-14 05:05:58,647||INFO|at
java.lang.reflect.Method.invoke(Method.java:601)
2013-11-14 05:05:58,647||INFO|at
org.apache.hadoop.util.RunJar.main(RunJar.java:212)
2013-11-14 05:05:58,662||INFO|RUNNING: /usr/bin/hdfs dfs -ls
tezSimpleSplitClientInput
2013-11-14 05:06:00,239||INFO|RUNNING: /usr/bin/hdfs dfs
-rm -r -skipTrash tezSimpleSplitClientInput
2013-11-14 05:06:01,702||INFO|Deleted
tezSimpleSplitClientInput
2013-11-14 05:06:01,819||INFO|RUNNING: /usr/bin/hdfs dfs -ls
tezSimpleSplitClientInputOutput
2013-11-14 05:06:03,365|main|INFO|TEST
"testSimpleSplit[SimpleSplitClient]" FAILED in 13.02 seconds