Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4519

Support override of jsvc binary and log file locations when launching secure datanode.

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.2.0, 3.0.0-alpha1
    • 1.2.0, 2.1.0-beta
    • datanode, scripts
    • None
    • Reviewed
    • Hide
      With this improvement the following options are available in release 1.2.0 and later on 1.x release stream:
      1. jsvc location can be overridden by setting environment variable JSVC_HOME. Defaults to jsvc binary packaged within the Hadoop distro.
      2. jsvc log output is directed to the file defined by JSVC_OUTFILE. Defaults to $HADOOP_LOG_DIR/jsvc.out.
      3. jsvc error output is directed to the file defined by JSVC_ERRFILE file. Defaults to $HADOOP_LOG_DIR/jsvc.err.

      With this improvement the following options are available in release 2.0.4 and later on 2.x release stream:
      1. jsvc log output is directed to the file defined by JSVC_OUTFILE. Defaults to $HADOOP_LOG_DIR/jsvc.out.
      2. jsvc error output is directed to the file defined by JSVC_ERRFILE file. Defaults to $HADOOP_LOG_DIR/jsvc.err.

      For overriding jsvc location on 2.x releases, here is the release notes from HDFS-2303:
      To run secure Datanodes users must install jsvc for their platform and set JSVC_HOME to point to the location of jsvc in their environment.
      Show
      With this improvement the following options are available in release 1.2.0 and later on 1.x release stream: 1. jsvc location can be overridden by setting environment variable JSVC_HOME. Defaults to jsvc binary packaged within the Hadoop distro. 2. jsvc log output is directed to the file defined by JSVC_OUTFILE. Defaults to $HADOOP_LOG_DIR/jsvc.out. 3. jsvc error output is directed to the file defined by JSVC_ERRFILE file. Defaults to $HADOOP_LOG_DIR/jsvc.err. With this improvement the following options are available in release 2.0.4 and later on 2.x release stream: 1. jsvc log output is directed to the file defined by JSVC_OUTFILE. Defaults to $HADOOP_LOG_DIR/jsvc.out. 2. jsvc error output is directed to the file defined by JSVC_ERRFILE file. Defaults to $HADOOP_LOG_DIR/jsvc.err. For overriding jsvc location on 2.x releases, here is the release notes from HDFS-2303 : To run secure Datanodes users must install jsvc for their platform and set JSVC_HOME to point to the location of jsvc in their environment.

    Description

      Currently, builds based on branch-1 bundle a specific version of jsvc pre-built for Linux, and the startup scripts hard-code the location of the output and error files. Some deployments may prefer to upgrade to a different version of jsvc, independent of the version bundled in Hadoop, and redirect its output elsewhere.

      Attachments

        1. HDFS-4519.1.patch
          1 kB
          Chris Nauroth
        2. HDFS-4519-branch-1.1.patch
          3 kB
          Chris Nauroth

        Issue Links

          Activity

            atm Aaron Myers added a comment -

            Hey Chris, is there not a similar issue in branch-2? There may very well not be - just checking.

            atm Aaron Myers added a comment - Hey Chris, is there not a similar issue in branch-2? There may very well not be - just checking.
            cnauroth Chris Nauroth added a comment -

            trunk already has the capability to set the JSVC_HOME environment variable to point at the operator's preferred version of jsvc. Part of this change will back-port support for the JSVC_HOME environment variable to branch-1. Unlike trunk, we will maintain the behavior of defaulting to Hadoop's bundled version of jsvc for backwards-compatibility if JSVC_HOME is not specified. Additionally, we will introduce JSVC_OUTFILE and JSVC_ERRFILE for redirecting the output and error streams of jsvc.

            One specific usage of this is to allow use of a new feature added in recent jsvc versions to redirect outfile and errfile to stdout and stderr via the special arguments '&1' and '&2' respectively. This feature is not supported in the current version bundled in Hadoop, and commons-daemon has stopped providing pre-built binary distributions, so we cannot upgrade the bundled version.

            cnauroth Chris Nauroth added a comment - trunk already has the capability to set the JSVC_HOME environment variable to point at the operator's preferred version of jsvc. Part of this change will back-port support for the JSVC_HOME environment variable to branch-1. Unlike trunk, we will maintain the behavior of defaulting to Hadoop's bundled version of jsvc for backwards-compatibility if JSVC_HOME is not specified. Additionally, we will introduce JSVC_OUTFILE and JSVC_ERRFILE for redirecting the output and error streams of jsvc. One specific usage of this is to allow use of a new feature added in recent jsvc versions to redirect outfile and errfile to stdout and stderr via the special arguments '&1' and '&2' respectively. This feature is not supported in the current version bundled in Hadoop, and commons-daemon has stopped providing pre-built binary distributions, so we cannot upgrade the bundled version.
            cnauroth Chris Nauroth added a comment -

            The attached patch introduces JSVC_HOME, JSVC_OUTFILE, and JSVC_ERRFILE environment variables in the hadoop script on branch-1. I've successfully tested this change manually on a secure cluster. I tested overriding to a manually built commons-daemon-1.0.13, and I also tested that the default behavior of using the bundled version of jsvc is still intact.

            Here is an example of launching a secure data node using this feature:

            JSVC_HOME=/home/cnauroth/commons-daemon-1.0.13-src/src/native/unix \
              HADOOP_CLASSPATH=/home/cnauroth/commons-daemon-1.0.13-src/dist/commons-daemon-1.0.13.jar \
              HADOOP_USER_CLASSPATH_FIRST=true \
              JSVC_OUTFILE='&1' \
              JSVC_ERRFILE='&2' \
              HADOOP_SECURE_DN_USER=cnauroth \
              sudo -E bin/hadoop datanode
            
            cnauroth Chris Nauroth added a comment - The attached patch introduces JSVC_HOME, JSVC_OUTFILE, and JSVC_ERRFILE environment variables in the hadoop script on branch-1. I've successfully tested this change manually on a secure cluster. I tested overriding to a manually built commons-daemon-1.0.13, and I also tested that the default behavior of using the bundled version of jsvc is still intact. Here is an example of launching a secure data node using this feature: JSVC_HOME=/home/cnauroth/commons-daemon-1.0.13-src/src/ native /unix \ HADOOP_CLASSPATH=/home/cnauroth/commons-daemon-1.0.13-src/dist/commons-daemon-1.0.13.jar \ HADOOP_USER_CLASSPATH_FIRST= true \ JSVC_OUTFILE= '&1' \ JSVC_ERRFILE= '&2' \ HADOOP_SECURE_DN_USER=cnauroth \ sudo -E bin/hadoop datanode
            cnauroth Chris Nauroth added a comment -

            Hi, Aaron.

            I don't see an exact match for this issue, based on the comments I gave above. I see HDFS-4497, which upgrades the commons-daemon version dependency on the Java side to resolve a build issue. I've been watching and code reviewing that one. I also found HDFS-2350, an old jira you filed about how secure data nodes don't print to console. I reassigned HDFS-2350 to myself, because the patch I posted here will give us the capability to send jsvc outfile/errfile to stdout/stderr.

            If you see an existing issue that I missed though, please let me know. Thanks!

            cnauroth Chris Nauroth added a comment - Hi, Aaron. I don't see an exact match for this issue, based on the comments I gave above. I see HDFS-4497 , which upgrades the commons-daemon version dependency on the Java side to resolve a build issue. I've been watching and code reviewing that one. I also found HDFS-2350 , an old jira you filed about how secure data nodes don't print to console. I reassigned HDFS-2350 to myself, because the patch I posted here will give us the capability to send jsvc outfile/errfile to stdout/stderr. If you see an existing issue that I missed though, please let me know. Thanks!
            cnauroth Chris Nauroth added a comment -

            Relating to HDFS-2303, which shows the patch that introduced JSVC_HOME on trunk.

            cnauroth Chris Nauroth added a comment - Relating to HDFS-2303 , which shows the patch that introduced JSVC_HOME on trunk.
            josb-ebay Jos Backus added a comment -

            While this is an interesting intermediate solution (I suggested something similar to HWX when I tried adding daemontools support to the secure datanode), wouldn't it make more sense to retire the use of jsvc altogether? This would also allow the exception in the hadoop startup wrapper to support datanode secure mode to be removed.

            josb-ebay Jos Backus added a comment - While this is an interesting intermediate solution (I suggested something similar to HWX when I tried adding daemontools support to the secure datanode), wouldn't it make more sense to retire the use of jsvc altogether? This would also allow the exception in the hadoop startup wrapper to support datanode secure mode to be removed.
            cnauroth Chris Nauroth added a comment -

            Hi, Jos. I'd like to suggest that we keep the scope of this jira limited to overriding the jsvc binary, the outfile, and the errfile. Since jsvc is the expected way to launch a secure data node right now, a change that retires it may have to be flagged incompatible. The patch I attached here is fairly small and fully backwards-compatible with the existing behavior.

            I'd definitely be curious to see what you have in mind though if you want to file another jira. Are you thinking that we should drop the jsvc dependency from Hadoop and push the process management responsibility to the operator, who can choose to use jsvc or daemontools or whatever other tool is preferred by that operator?

            cnauroth Chris Nauroth added a comment - Hi, Jos. I'd like to suggest that we keep the scope of this jira limited to overriding the jsvc binary, the outfile, and the errfile. Since jsvc is the expected way to launch a secure data node right now, a change that retires it may have to be flagged incompatible. The patch I attached here is fairly small and fully backwards-compatible with the existing behavior. I'd definitely be curious to see what you have in mind though if you want to file another jira. Are you thinking that we should drop the jsvc dependency from Hadoop and push the process management responsibility to the operator, who can choose to use jsvc or daemontools or whatever other tool is preferred by that operator?
            josb-ebay Jos Backus added a comment -

            Sorry, Chris, didn't mean to derail this ticket. I like what you propose regarding fixing/extending the jsvc support, so +1 from me

            I will file a different ticket for my enhancement suggestion (just like I did internally when the issue came up). Briefly, my suggestion was to merge the jsvc-run class code into the main datanode class, and use JNI to call setuid() to drop privileges after allocating the privilege-requiring resources (see http://www2.sys-con.com/itsg/virtualcd/java/archives/0510/Silverman/index.html for details how this can be done).

            josb-ebay Jos Backus added a comment - Sorry, Chris, didn't mean to derail this ticket. I like what you propose regarding fixing/extending the jsvc support, so +1 from me I will file a different ticket for my enhancement suggestion (just like I did internally when the issue came up). Briefly, my suggestion was to merge the jsvc-run class code into the main datanode class, and use JNI to call setuid() to drop privileges after allocating the privilege-requiring resources (see http://www2.sys-con.com/itsg/virtualcd/java/archives/0510/Silverman/index.html for details how this can be done).
            cnauroth Chris Nauroth added a comment -

            Also attaching trunk patch for feature parity. trunk already has JSVC_HOME, so I just needed to add JSVC_OUTFILE and JSVC_ERRFILE.

            cnauroth Chris Nauroth added a comment - Also attaching trunk patch for feature parity. trunk already has JSVC_HOME, so I just needed to add JSVC_OUTFILE and JSVC_ERRFILE.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12571253/HDFS-4519.1.patch
            against trunk revision .

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no new tests are needed for this patch.
            Also please list what manual steps were performed to verify this patch.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 eclipse:eclipse. The patch built with eclipse:eclipse.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

            +1 contrib tests. The patch passed contrib unit tests.

            Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4014//testReport/
            Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4014//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12571253/HDFS-4519.1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4014//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4014//console This message is automatically generated.
            cnauroth Chris Nauroth added a comment -

            Jenkins gave -1 for no new tests, because this patch changes only shell scripts. I have manually tested both the trunk and branch-1 patches.

            cnauroth Chris Nauroth added a comment - Jenkins gave -1 for no new tests, because this patch changes only shell scripts. I have manually tested both the trunk and branch-1 patches.

            +1 for the trunk patch.

            sureshms Suresh Srinivas added a comment - +1 for the trunk patch.

            +1 for the branch-1 patch.

            sureshms Suresh Srinivas added a comment - +1 for the branch-1 patch.
            hudson Hudson added a comment -

            Integrated in Hadoop-trunk-Commit #3419 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3419/)
            HDFS-4519. Support overriding jsvc binary and log file locations when launching secure datanode. Contributed by Chris Nauroth. (Revision 1453050)

            Result = SUCCESS
            suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1453050
            Files :

            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
            hudson Hudson added a comment - Integrated in Hadoop-trunk-Commit #3419 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3419/ ) HDFS-4519 . Support overriding jsvc binary and log file locations when launching secure datanode. Contributed by Chris Nauroth. (Revision 1453050) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1453050 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs

            I committed the patch to trunk, branch-2 and branch-1. Thank you Chris!

            sureshms Suresh Srinivas added a comment - I committed the patch to trunk, branch-2 and branch-1. Thank you Chris!
            hudson Hudson added a comment -

            Integrated in Hadoop-Yarn-trunk #147 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/147/)
            HDFS-4519. Support overriding jsvc binary and log file locations when launching secure datanode. Contributed by Chris Nauroth. (Revision 1453050)

            Result = SUCCESS
            suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1453050
            Files :

            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
            hudson Hudson added a comment - Integrated in Hadoop-Yarn-trunk #147 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/147/ ) HDFS-4519 . Support overriding jsvc binary and log file locations when launching secure datanode. Contributed by Chris Nauroth. (Revision 1453050) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1453050 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
            hudson Hudson added a comment -

            Integrated in Hadoop-Hdfs-trunk #1336 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1336/)
            HDFS-4519. Support overriding jsvc binary and log file locations when launching secure datanode. Contributed by Chris Nauroth. (Revision 1453050)

            Result = SUCCESS
            suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1453050
            Files :

            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
            hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1336 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1336/ ) HDFS-4519 . Support overriding jsvc binary and log file locations when launching secure datanode. Contributed by Chris Nauroth. (Revision 1453050) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1453050 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
            hudson Hudson added a comment -

            Integrated in Hadoop-Mapreduce-trunk #1364 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1364/)
            HDFS-4519. Support overriding jsvc binary and log file locations when launching secure datanode. Contributed by Chris Nauroth. (Revision 1453050)

            Result = SUCCESS
            suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1453050
            Files :

            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
            hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1364 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1364/ ) HDFS-4519 . Support overriding jsvc binary and log file locations when launching secure datanode. Contributed by Chris Nauroth. (Revision 1453050) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1453050 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
            mattf Matthew Foley added a comment -

            Closed upon release of Hadoop 1.2.0.

            mattf Matthew Foley added a comment - Closed upon release of Hadoop 1.2.0.

            People

              cnauroth Chris Nauroth
              cnauroth Chris Nauroth
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: