Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3578

starting nodemanager as 'root' gives "Unknown -jvm option"

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 2.0.0-alpha
    • Component/s: nodemanager
    • Labels:
      None
    • Environment:

      jdk 1.6.0.26, ubuntu 11.04

    • Hadoop Flags:
      Reviewed

      Description

      running "sudo HADOOP_ROOT/bin/yarn-daemon.sh start nodemanager" I get "unknown -jvm option" (jdk version 1.6.0.26). The problem seems to be with line 204 in yarn:

      elif [ "$COMMAND" = "nodemanager" ] ; then
      CLASSPATH=$

      {CLASSPATH}

      :$YARN_CONF_DIR/nm-config/log4j.properties
      CLASS='org.apache.hadoop.yarn.server.nodemanager.NodeManager'
      if [[ $EUID -eq 0 ]]; then
      YARN_OPTS="$YARN_OPTS -jvm server $YARN_NODEMANAGER_OPTS"
      else
      YARN_OPTS="$YARN_OPTS -server $YARN_NODEMANAGER_OPTS"
      fi

      using -server seems to solve the problem for me.

      I tested using build 929 from https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk/.

        Activity

        Hide
        Allen Wittenauer added a comment -

        Chances are high that your jsvc is busted in some way or you are running in secure mode or something similar. It might just need to be recompiled the proper bitness. In any case "-jvm server" is a jsvc option used only when the daemon is run as root, thus why it is required for $EUID = 0. "-server" is a JVM option and used when the daemon is run by not root.

        Show
        Allen Wittenauer added a comment - Chances are high that your jsvc is busted in some way or you are running in secure mode or something similar. It might just need to be recompiled the proper bitness. In any case "-jvm server" is a jsvc option used only when the daemon is run as root, thus why it is required for $EUID = 0. "-server" is a JVM option and used when the daemon is run by not root.
        Show
        Eli Collins added a comment - These two jiras are relevant: https://issues.cloudera.org/browse/DISTRO-252 https://issues.apache.org/jira/browse/DAEMON-208
        Hide
        Vinod Kumar Vavilapalli added a comment -

        It is not advisable to start NodeManagers as root for security purposes. That applies for all of yarn/mapreduce daemons. Not sure why it was allowed to run as root. Could be a copy-paste bug.

        If we allow root, he bug is still valid I suppose.

        BTW, none of yarn daemons use jsvc.

        Show
        Vinod Kumar Vavilapalli added a comment - It is not advisable to start NodeManagers as root for security purposes. That applies for all of yarn/mapreduce daemons. Not sure why it was allowed to run as root. Could be a copy-paste bug. If we allow root, he bug is still valid I suppose. BTW, none of yarn daemons use jsvc.
        Hide
        Tom White added a comment -

        Patch to remove jsvc options since jsvc isn't used by node managers (as Vinod pointed out).

        Show
        Tom White added a comment - Patch to remove jsvc options since jsvc isn't used by node managers (as Vinod pointed out).
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12517138/MAPREDUCE-3578.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.mapred.TestJobClientGetJob
        org.apache.hadoop.mapred.TestMRWithDistributedCache
        org.apache.hadoop.mapred.TestLocalModeWithNewApis

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2005//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2005//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517138/MAPREDUCE-3578.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapred.TestJobClientGetJob org.apache.hadoop.mapred.TestMRWithDistributedCache org.apache.hadoop.mapred.TestLocalModeWithNewApis +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2005//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2005//console This message is automatically generated.
        Hide
        Alejandro Abdelnur added a comment -

        +1

        Show
        Alejandro Abdelnur added a comment - +1
        Hide
        Tom White added a comment -

        I've just committed this.

        Show
        Tom White added a comment - I've just committed this.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1932 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1932/)
        MAPREDUCE-3578. Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941)

        Result = SUCCESS
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1932 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1932/ ) MAPREDUCE-3578 . Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1857 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1857/)
        MAPREDUCE-3578. Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941)

        Result = SUCCESS
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1857 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1857/ ) MAPREDUCE-3578 . Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #656 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/656/)
        Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942)

        Result = SUCCESS
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #656 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/656/ ) Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #665 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/665/)
        Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942)

        Result = SUCCESS
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #665 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/665/ ) Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1866 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1866/)
        MAPREDUCE-3578. Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941)

        Result = ABORTED
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1866 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1866/ ) MAPREDUCE-3578 . Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941) Result = ABORTED tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #673 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/673/)
        Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942)

        Result = ABORTED
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #673 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/673/ ) Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942) Result = ABORTED tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #193 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/193/)
        Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942)

        Result = UNSTABLE
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #193 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/193/ ) Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942) Result = UNSTABLE tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #980 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/980/)
        MAPREDUCE-3578. Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941)

        Result = SUCCESS
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #980 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/980/ ) MAPREDUCE-3578 . Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #221 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/221/)
        Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942)

        Result = FAILURE
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #221 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/221/ ) Merge -r 1298940:1298941 from trunk to branch-0.23. Fixes: MAPREDUCE-3578 (Revision 1298942) Result = FAILURE tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298942 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1015 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1015/)
        MAPREDUCE-3578. Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941)

        Result = SUCCESS
        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1015 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1015/ ) MAPREDUCE-3578 . Starting nodemanager as root gives "Unknown -jvm option" (Revision 1298941) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1298941 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/bin/yarn

          People

          • Assignee:
            Tom White
            Reporter:
            Gilad Wolff
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development