Hadoop Common
  1. Hadoop Common
  2. HADOOP-8010

hadoop-config.sh spews error message when HADOOP_HOME_WARN_SUPPRESS is set to true and HADOOP_HOME is present

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.0.1
    • Component/s: scripts
    • Labels:
      None

      Description

      Running hadoop daemon commands when HADOOP_HOME_WARN_SUPPRESS is set to true and HADOOP_HOME is present produces:

        [: 76: true: unexpected operator
      
      1. HADOOP-8010.patch.txt
        0.5 kB
        Roman Shaposhnik

        Activity

        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12512701/HADOOP-8010.patch.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/552//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12512701/HADOOP-8010.patch.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/552//console This message is automatically generated.
        Hide
        Eli Collins added a comment -

        I think this is because you're using sh (or dash) instead of bash. In bash "=" and "==" are equivalent.

        Show
        Eli Collins added a comment - I think this is because you're using sh (or dash) instead of bash. In bash "=" and "==" are equivalent.
        Hide
        Eli Collins added a comment -

        Adding env bash to hadoop-config.sh should fix this but given that the scripts that source hadoop-config.sh explicitly use env bash this shouldn't be necessary.

        Show
        Eli Collins added a comment - Adding env bash to hadoop-config.sh should fix this but given that the scripts that source hadoop-config.sh explicitly use env bash this shouldn't be necessary.
        Hide
        Matt Foley added a comment -

        +1 for code review. Frankly I'm not clear why the previous version had this error on some systems, and I can't reproduce it on my RHEL box. But the proposed patch brings the boolean test into POSIX conformance and is certainly acceptable.

        Committing to 1.0.1 at Roman's request.

        Show
        Matt Foley added a comment - +1 for code review. Frankly I'm not clear why the previous version had this error on some systems, and I can't reproduce it on my RHEL box. But the proposed patch brings the boolean test into POSIX conformance and is certainly acceptable. Committing to 1.0.1 at Roman's request.
        Hide
        Matt Foley added a comment -

        Committed to branch-1 and branch-1.0.

        Show
        Matt Foley added a comment - Committed to branch-1 and branch-1.0.
        Hide
        Eli Collins added a comment -

        We use "==" elsewhere in bin/hadoop* shouldn't those be fixed as well then?

        I think there's another bug lurking here, == is obviously correct in bash.

        hadoop-branch-1 $ bash bin/hadoop-config.sh 
        hadoop-branch-1 $ dash bin/hadoop-config.sh 
        [: 76: unexpected operator
        hadoop-branch-1 $ 
        
        Show
        Eli Collins added a comment - We use "==" elsewhere in bin/hadoop* shouldn't those be fixed as well then? I think there's another bug lurking here, == is obviously correct in bash. hadoop-branch-1 $ bash bin/hadoop-config.sh hadoop-branch-1 $ dash bin/hadoop-config.sh [: 76: unexpected operator hadoop-branch-1 $
        Hide
        Roman Shaposhnik added a comment -

        @Matt, thank a lot for committing it!

        @Eli, the difference between hadoop-config.sh and everything else is that hadoop-config.sh doesn't control which scripts are sourcing it (it is meant to be sourced not executed) and thus needs to be as POSIX compliant as possible. The rest of the scripts where '==' is used in Hadoop 1.0 branch are all specifying #!/usr/bin/env bash so there's no issue.

        Show
        Roman Shaposhnik added a comment - @Matt, thank a lot for committing it! @Eli, the difference between hadoop-config.sh and everything else is that hadoop-config.sh doesn't control which scripts are sourcing it (it is meant to be sourced not executed) and thus needs to be as POSIX compliant as possible. The rest of the scripts where '==' is used in Hadoop 1.0 branch are all specifying #!/usr/bin/env bash so there's no issue.
        Hide
        Eli Collins added a comment -

        Right, but both the daemons explicitly specify bash before sourcing hadoop-config.sh, so how are they triggering this?

        Show
        Eli Collins added a comment - Right, but both the daemons explicitly specify bash before sourcing hadoop-config.sh, so how are they triggering this?
        Hide
        Roman Shaposhnik added a comment -

        hadoop daemon scripts are not the only thing that sources hadoop-config.sh

        Of course, one can argue that it doesn't constitute a stable API and hence shouldn't be sourced outside of Hadoop tree. That said, in this particular case it was easier to keep hadoop-config.sh free from bash'isms.

        Show
        Roman Shaposhnik added a comment - hadoop daemon scripts are not the only thing that sources hadoop-config.sh Of course, one can argue that it doesn't constitute a stable API and hence shouldn't be sourced outside of Hadoop tree. That said, in this particular case it was easier to keep hadoop-config.sh free from bash'isms.
        Hide
        Eli Collins added a comment -

        The change is fine by me, I'm just trying to understand it.

        The bug is that running the hadoop daemon commands hits this error, but given that the error should only hit when not using bash, and the hadoop daemons explicitly use bash, I suspect there's something else going on here. Eg is this because bash is a symlink to dash on debian?

        Show
        Eli Collins added a comment - The change is fine by me, I'm just trying to understand it. The bug is that running the hadoop daemon commands hits this error, but given that the error should only hit when not using bash, and the hadoop daemons explicitly use bash, I suspect there's something else going on here. Eg is this because bash is a symlink to dash on debian?

          People

          • Assignee:
            Roman Shaposhnik
            Reporter:
            Roman Shaposhnik
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development