Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5459

Update the doc of running MRv1 examples jar on YARN

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2.0
    • Component/s: None
    • Labels:
      None

      Description

      In addition to adding two env vars: HADOOP_USER_CLASSPATH_FIRST and HADOOP_CLASSPATH, we still need to add

          <property>
              <name>mapreduce.job.user.classpath.first</name>
              <value>true</value>
          </property>
      

      in mapred-site.xml to make sure that the MRv1 examples jar runs correctly on YARN. Some examples will use Java reflection to find the classes in the examples jar dynamically when they are running. With this configuration, the MRv1 examples jar will appear before the MRv2 examples jar in CLASSPATH of the processes in YARN containers. Therefore, the classes found via reflection will be picked from MRv1 examples jar instead of MRv2 examples jar as well.

      MapReduce_Compatibility_Hadoop1_Hadoop2.apt.vm needs to be updated to document this.

        Attachments

        1. MAPREDUCE-5459.1.patch
          2 kB
          Zhijie Shen

          Issue Links

            Activity

              People

              • Assignee:
                zjshen Zhijie Shen
                Reporter:
                zjshen Zhijie Shen
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: