Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12574

Multiple Xmx Parameters to Java Process

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 2.7.1
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      When launching hadoop, the setting of environment variable HADOOP_HEAPMAX results in multiple Xmx parameters being passed to the java application launcher. This is because a default Xmx setting included in the HADOOP_CLIENT_OPTS in etc/hadoop/hadoop-env.sh, line 61:

      # The following applies to multiple commands (fs, dfs, fsck, distcp etc)
      export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"
      

      is merged into the variable HADOOP_OPTS in bin/hadoop, line 160:

          # Always respect HADOOP_OPTS and HADOOP_CLIENT_OPTS
          HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"
      

      Orthogonally, the HADOOP_HEAPMAX environment variable triggers the setting of JAVA_HEAP_MAX in hadoop-config.sh, line 192:

      # check envvars which might override default args
      if [ "$HADOOP_HEAPSIZE" != "" ]; then
        #echo "run with heapsize $HADOOP_HEAPSIZE"
        JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m"
        #echo $JAVA_HEAP_MAX
      fi
      

      Which is ultimately passed to the java application launcher in bin/hadoop, line 166:

          exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
      

      The result: two Xmx settings passed to the Java application launcher. Which one is respected depends on the JVM, but for HotSpot the rightmost setting is respected, and so the HADOOP_HEAPMAX environment variable has no effect and the default setting in HADOOP_CLIENT_OPTS wins.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                Ellitron Jonathan Ellithorpe
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: