Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6590

Update MRJobConfig.DEFAULT_MAPRED_ADMIN_USER_ENV value for loading correct navite lib when using ubertask feature

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 2.7.1
    • Fix Version/s: None
    • Component/s: mr-am
    • Labels:
      None

      Description

      By default, the MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set.
      If yarn.app.mapreduce.am.admin.user.env (or yarn.app.mapreduce.am.env) is not configured to set LD_LIBRARY_PATH, MR AM will fail to load the native library, then you can find the error message as below.

      org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable.

      I found the patch in https://issues.apache.org/jira/browse/MAPREDUCE-5799 update YARNRunner code and use MRJobConfig.DEFAULT_MAPRED_ADMIN_USER_ENV as the default value. The code is as below.

      YARNRunner.java
          MRApps.setEnvFromInputString(environment, 
              conf.get(MRJobConfig.MR_AM_ADMIN_USER_ENV,
                  MRJobConfig.DEFAULT_MAPRED_ADMIN_USER_ENV), conf);
      

      In fact the parameters yarn.app.mapreduce.am.env and yarn.app.mapreduce.am.admin.user.env are resolved in YARNRunner running on hadoop client host.
      But their values, e.g. LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native, will be used on hadoop cluster hosts.
      If the hadoop client host installs hadoop on a different path from that on hadoop cluster hosts,
      the $HADOOP_COMMON_HOME/lib/native will be resolved to a path which is not existed on hadoop cluster hosts.
      Then MRAppMaster running on hadoop cluster host cannot load native lib.
      If don't use MAPREDUCE-5799 patch, I set my hadoop client's mapred-site.xml with the following content.

      <property>
              <name>yarn.app.mapreduce.am.admin.user.env</name>
              <value>LD_LIBRARY_PATH={{HADOOP_COMMON_HOME}}/lib/native</value>        
      </property>
      

      In this way, the YARNRunner will put LD_LIBRARY_PATH=HADOOP_COMMON_HOME/lib/native into environment.
      The method ContainerLaunch.expandEnvironment running in NodeManager can translate LD_LIBRARY_PATH=HADOOP_COMMON_HOME/lib/native
      to LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native.
      The host running NodeManager can find its $HADOOP_COMMON_HOME/lib/native path.

      I suggest MRJobConfig.DEFAULT_MAPRED_ADMIN_USER_ENV should be defined as below.

      MRJobConfig.java
      public final String DEFAULT_MAPRED_ADMIN_USER_ENV = 
            Shell.WINDOWS ? 
                "PATH={{PATH}};{{HADOOP_COMMON_HOME}}\\bin":
                "LD_LIBRARY_PATH={{HADOOP_COMMON_HOME}}/lib/native";
      

      Please see my patch.

        Attachments

        1. MAPREDUCE-6590.patch
          2 kB
          NING DING

          Issue Links

            Activity

              People

              • Assignee:
                iceberg565 NING DING
                Reporter:
                iceberg565 NING DING
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: