Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
None
-
None
-
None
Description
Following the installation instructions at: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
the randomwriter example runs successfully. However, other full map & reduce jobs (e.g. wordcount) fail with the error:
java.lang.UnsupportedOperationException: Incompatible with LocalRunner
at org.apache.hadoop.mapred.YarnOutputFiles.getInputFile(YarnOutputFiles.java:200)
at org.apache.hadoop.mapred.ReduceTask.getMapFiles(ReduceTask.java:223)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:148)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1094)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:143)
The ReduceTask evaluates the isLocal flag based on the property "mapreduce.jobtracker.address", the default value for this property in mapred-default.xml is 'local' and this is the cause of the problem.
Setting "mapreduce.jobtracker.address" in the mapred-site.xml to something other than "local" seems to solve the problem.