Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1943

fail to start datanode while start-dfs.sh is executed by root user

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0, 0.22.0, 0.23.0
    • Fix Version/s: 0.22.0, 0.23.0, 1.0.0
    • Component/s: scripts
    • Labels:
      None

      Description

      When start-dfs.sh is run by root user, we got the following error message:

      1. start-dfs.sh
        Starting namenodes on [localhost ]
        localhost: namenode running as process 2556. Stop it first.
        localhost: starting datanode, logging to /usr/hadoop/hadoop-common-0.23.0-SNAPSHOT/bin/../logs/hadoop-root-datanode-cspf01.out
        localhost: Unrecognized option: -jvm
        localhost: Could not create the Java virtual machine.

      The -jvm options should be passed to jsvc when we starting a secure
      datanode, but it still passed to java when start-dfs.sh is run by root
      while secure datanode is disabled. This is a bug of bin/hdfs.

      1. HDFS-1943.patch
        0.5 kB
        Wei Yongjun
      2. HDFS-1943-branch-0.20-security.patch
        0.5 kB
        Matt Foley

        Issue Links

          Activity

          Roman Shaposhnik made changes -
          Attachment 0001-HDFS-1943.-fail-to-start-datanode-while-start-dfs.sh.patch [ 12559788 ]
          Roman Shaposhnik made changes -
          Attachment 0001-HDFS-1943.-fail-to-start-datanode-while-start-dfs.sh.patch [ 12559788 ]
          Steve Loughran made changes -
          Link This issue is related to BIGTOP-683 [ BIGTOP-683 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Matt Foley made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Target Version/s 0.23.0, 0.22.0, 0.20.205.1 [ 12315571, 12314241, 12318243 ] 0.20.205.1, 0.22.0, 0.23.0 [ 12318243, 12314241, 12315571 ]
          Fix Version/s 0.20.205.1 [ 12318243 ]
          Resolution Fixed [ 1 ]
          Hide
          Matt Foley added a comment -

          Committed to branch-0.20-security and branch-0.20-security-205.

          Show
          Matt Foley added a comment - Committed to branch-0.20-security and branch-0.20-security-205.
          Hide
          Roman Shaposhnik added a comment -

          Matt, this works for me. Please go ahead and commit.

          Show
          Roman Shaposhnik added a comment - Matt, this works for me. Please go ahead and commit.
          Matt Foley made changes -
          Assignee Wei Yongjun [ weiyj ] Matt Foley [ mattf ]
          Matt Foley made changes -
          Attachment HDFS-1943-branch-0.20-security.patch [ 12502583 ]
          Hide
          Matt Foley added a comment -

          This would seem to be the appropriate back-port of the one-line patch.

          Roman, can you test it please?

          Show
          Matt Foley added a comment - This would seem to be the appropriate back-port of the one-line patch. Roman, can you test it please?
          Matt Foley made changes -
          Target Version/s 0.20.205.1, 0.22.0, 0.23.0 [ 12318243, 12314241, 12315571 ]
          Matt Foley made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Hide
          Matt Foley added a comment -

          Reopened for back-port to 0.20.205.1, per request from Roman Shaposhnik.

          Show
          Matt Foley added a comment - Reopened for back-port to 0.20.205.1, per request from Roman Shaposhnik.
          Roman Shaposhnik made changes -
          Affects Version/s 0.20.205.0 [ 12316392 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #90 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/90/)
          HDFS-1943. fail to start datanode while start-dfs.sh is executed by root user. Contributed by Wei Yongjun.

          cos : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176682
          Files :

          • /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt
          • /hadoop/common/branches/branch-0.22/hdfs/bin/hdfs
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #90 (See https://builds.apache.org/job/Hadoop-Hdfs-22-branch/90/ ) HDFS-1943 . fail to start datanode while start-dfs.sh is executed by root user. Contributed by Wei Yongjun. cos : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1176682 Files : /hadoop/common/branches/branch-0.22/hdfs/CHANGES.txt /hadoop/common/branches/branch-0.22/hdfs/bin/hdfs
          Konstantin Boudnik made changes -
          Link This issue blocks HDFS-2377 [ HDFS-2377 ]
          Konstantin Boudnik made changes -
          Link This issue is duplicated by HDFS-2377 [ HDFS-2377 ]
          Konstantin Boudnik made changes -
          Link This issue blocks HDFS-2377 [ HDFS-2377 ]
          Konstantin Boudnik made changes -
          Fix Version/s 0.22.0 [ 12314241 ]
          Affects Version/s 0.22.0 [ 12314241 ]
          Konstantin Shvachko made changes -
          Link This issue is cloned as HDFS-2377 [ HDFS-2377 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #678 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/678/)
          HDFS-1943. Fail to start datanode while start-dfs.sh is executed by root user. Contributed by Wei Yongjun.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1127706
          Files :

          • /hadoop/hdfs/trunk/bin/hdfs
          • /hadoop/hdfs/trunk/CHANGES.txt
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #678 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/678/ ) HDFS-1943 . Fail to start datanode while start-dfs.sh is executed by root user. Contributed by Wei Yongjun. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1127706 Files : /hadoop/hdfs/trunk/bin/hdfs /hadoop/hdfs/trunk/CHANGES.txt
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #685 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/685/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #685 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/685/ )
          Jakob Homan made changes -
          Fix Version/s 0.23.0 [ 12315571 ]
          Fix Version/s Edit log branch (HDFS-1073) [ 12316334 ]
          Jakob Homan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Assignee Wei Yongjun [ weiyj ]
          Fix Version/s Edit log branch (HDFS-1073) [ 12316334 ]
          Fix Version/s 0.23.0 [ 12315571 ]
          Resolution Fixed [ 1 ]
          Hide
          Jakob Homan added a comment -

          Committed to trunk. Resolving. Thanks, Wei!

          Show
          Jakob Homan added a comment - Committed to trunk. Resolving. Thanks, Wei!
          Hide
          Jakob Homan added a comment -

          It's not a good idea to run the dn as root, but this shouldn't be the way it's prevented. Unit test failures can't be related. Committing.

          Show
          Jakob Homan added a comment - It's not a good idea to run the dn as root, but this shouldn't be the way it's prevented. Unit test failures can't be related. Committing.
          Hide
          Jakob Homan added a comment -

          +1

          Show
          Jakob Homan added a comment - +1
          Hide
          Bharath Mundlapudi added a comment -

          +1 to this patch.

          Show
          Bharath Mundlapudi added a comment - +1 to this patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12479293/HDFS-1943.patch
          against trunk revision 1103970.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:
          org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
          org.apache.hadoop.hdfs.TestFileConcurrentReader
          org.apache.hadoop.tools.TestJMXGet

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/531//testReport/
          Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/531//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/531//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12479293/HDFS-1943.patch against trunk revision 1103970. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestDFSStorageStateRecovery org.apache.hadoop.hdfs.TestFileConcurrentReader org.apache.hadoop.tools.TestJMXGet +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/531//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/531//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/531//console This message is automatically generated.
          Wei Yongjun made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Wei Yongjun made changes -
          Field Original Value New Value
          Attachment HDFS-1943.patch [ 12479293 ]
          Hide
          Wei Yongjun added a comment -

          patch to fix it.

          Show
          Wei Yongjun added a comment - patch to fix it.
          Wei Yongjun created issue -

            People

            • Assignee:
              Matt Foley
              Reporter:
              Wei Yongjun
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development