Hadoop Common
  1. Hadoop Common
  2. HADOOP-6851

Fix '$bin' path duplication in setup scripts

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Trivial Trivial
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: scripts
    • Labels:
      None

      Description

      I have my bash environment setup to echo absolute pathnames when a relative one is specified in 'cd'. This caused problems with all the Hadoop bash scripts because the script accidentally sets the $bin variable twice in this setup. (e.g. would set $bin="/path/bin/hadoop\n/path/bin/hadoop").

      This jira is for common scripts. I filed a separate jira for HDFS scripts, which share the same pattern.

      1. HADOOP-6851-0.22.patch
        2 kB
        Nicolas Spiegelberg

        Issue Links

          Activity

          Hide
          Eli Collins added a comment -

          Isn't this a dupe of HADOOP-6101? I don't see how the bin scripts hit this since they don't appear to use relative paths.

          Show
          Eli Collins added a comment - Isn't this a dupe of HADOOP-6101 ? I don't see how the bin scripts hit this since they don't appear to use relative paths.
          Hide
          Lane Schwartz added a comment -

          This problem occurs when CDPATH is set and includes "." (the present working directory). It is unsafe to assume that cd will not print to standard output.

          The bash documentation states:

          If a non-empty directory name from CDPATH is used, or if ‘-’ is the first argument, and the directory change is successful, the absolute pathname of the new working directory is written to the standard output.

          Steps to recreate:

          1. export CDPATH=.
          2. Run any hadoop script that contains the offending bin=`cd "$bin"; pwd` line

          In this case, because CDPATH includes "." and the current value of $bin is "." an invocation of "cd" will cause the current absolute path to be printed to standard output. I have verified this problem occurs under these circumstances using bash 4.1.2(1) on Scientific Linux 6.2.

          The safest fix is to find all instances of the following:

          bin=`cd "$bin"; pwd`
          

          and replace the above with the following:

          bin=`cd "$bin" &> /dev/null; pwd`
          
          Show
          Lane Schwartz added a comment - This problem occurs when CDPATH is set and includes "." (the present working directory). It is unsafe to assume that cd will not print to standard output. The bash documentation states: If a non-empty directory name from CDPATH is used, or if ‘-’ is the first argument, and the directory change is successful, the absolute pathname of the new working directory is written to the standard output. Steps to recreate: 1. export CDPATH=. 2. Run any hadoop script that contains the offending bin=`cd "$bin"; pwd` line In this case, because CDPATH includes "." and the current value of $bin is "." an invocation of "cd" will cause the current absolute path to be printed to standard output. I have verified this problem occurs under these circumstances using bash 4.1.2(1) on Scientific Linux 6.2. The safest fix is to find all instances of the following: bin=`cd "$bin" ; pwd` and replace the above with the following: bin=`cd "$bin" &> /dev/ null ; pwd`
          Hide
          Jianbin Wei added a comment -

          Hi Lane, you may want to reattach a new patch and click the "submit patch" to kick off the automatic hadoop QA process. In my machine, I got patch apply failure.

          ⚡ dev-support/test-patch.sh ~/Downloads/HADOOP-6851-0.22.patch
          Running in developer mode

          ======================================================================
          ======================================================================
          Testing patch for HADOOP-6851-0.22.patch.
          ======================================================================
          ======================================================================

          Patch file /Users/jianbwei/Downloads/HADOOP-6851-0.22.patch copied to /tmp
          The patch does not appear to apply with p0 to p2
          PATCH APPLICATION FAILED

          -1 overall.

          -1 patch. The patch command could not apply the patch.

          ======================================================================
          ======================================================================
          Finished build.
          ======================================================================
          ======================================================================

          Show
          Jianbin Wei added a comment - Hi Lane, you may want to reattach a new patch and click the "submit patch" to kick off the automatic hadoop QA process. In my machine, I got patch apply failure. ⚡ dev-support/test-patch.sh ~/Downloads/ HADOOP-6851 -0.22.patch Running in developer mode ====================================================================== ====================================================================== Testing patch for HADOOP-6851 -0.22.patch. ====================================================================== ====================================================================== Patch file /Users/jianbwei/Downloads/ HADOOP-6851 -0.22.patch copied to /tmp The patch does not appear to apply with p0 to p2 PATCH APPLICATION FAILED -1 overall. -1 patch. The patch command could not apply the patch. ====================================================================== ====================================================================== Finished build. ====================================================================== ======================================================================
          Hide
          Lane Schwartz added a comment -

          What should I build my patch against? The latest svn, or a particular release?

          Show
          Lane Schwartz added a comment - What should I build my patch against? The latest svn, or a particular release?
          Hide
          Jianbin Wei added a comment -

          I am not sure. But I guess you can specify the version you patch against when submitting the patch. I found it is not compatible with trunk and just notice that your patch is for 0.22.

          Show
          Jianbin Wei added a comment - I am not sure. But I guess you can specify the version you patch against when submitting the patch. I found it is not compatible with trunk and just notice that your patch is for 0.22.

            People

            • Assignee:
              Unassigned
              Reporter:
              Nicolas Spiegelberg
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development