Description
Hadoop "join" example fails on Ignite with the error like this:
Out: class org.apache.ignite.IgniteCheckedException: class org.apache.ignite.IgniteCheckedException: null [14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2MapTask.run0(HadoopV2MapTask.java:102) [14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2Task.run(HadoopV2Task.java:55) [14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2TaskContext.run(HadoopV2TaskContext.java:266) [14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.runTask(HadoopRunnableTask.java:209) [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.call0(HadoopRunnableTask.java:144) [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask$1.call(HadoopRunnableTask.java:116) [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask$1.call(HadoopRunnableTask.java:114) [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2TaskContext.runAsJobOwner(HadoopV2TaskContext.java:573) [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.call(HadoopRunnableTask.java:114) [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.call(HadoopRunnableTask.java:46) [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecutorService$2.body(HadoopExecutorService.java:186) [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) [14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at java.lang.Thread.run(Thread.java:745) [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: Caused by: java.lang.NullPointerException [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.<init>(TaskAttemptContextImpl.java:49) [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.hadoop.mapreduce.lib.join.Parser$WNode.createRecordReader(Parser.java:348) [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.hadoop.mapreduce.lib.join.Parser$CNode.createRecordReader(Parser.java:486) [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.hadoop.mapreduce.lib.join.CompositeInputFormat.createRecordReader(CompositeInputFormat.java:143) [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2MapTask.run0(HadoopV2MapTask.java:69) [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out: ... 12 more [14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907 Out:
This is because org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2Context sets the job id and task id, but does not set task attempt id. In Hadoop this is done in method org.apache.hadoop.mapred.Task#localizeConfiguration .
Attachments
Issue Links
- is related to
-
IGNITE-4808 Add all Hadoop examples as Ignite unit tests with default multi-JVM execution mode
-
- Closed
-