Hadoop Common
  1. Hadoop Common
  2. HADOOP-6953

start-{dfs,mapred}.sh scripts fail if HADOOP_HOME is not set

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Duplicate
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.22.0
    • Component/s: scripts
    • Labels:
      None

      Description

      If the HADOOP_HOME environment variable is not set then the start and stop scripts for HDFS and MapReduce fail with "Hadoop common not found.". The start-all.sh and stop-all.sh scripts are not affected.

        Issue Links

          Activity

          Hide
          Zhenhua (Gerald) Guo added a comment -

          You can add following code to files "hdfs-config.sh" and "mapred-config.sh".

          this="$

          {BASH_SOURCE-$0}

          "
          while [ -h "$this" ]; do
          ls=`ls -ld "$this"`
          link=`expr "$ls" : '.-> (.)$'`
          if expr "$link" : './.' > /dev/null; then
          this="$link"
          else
          this=`dirname "$this"`/"$link"
          fi
          done

          1. convert relative path to absolute path
            common_bin=`dirname "$this"`
            script=`basename "$this"`
            common_bin=`cd "$common_bin"; pwd`
            this="$common_bin/$script"
          1. the root of the Hadoop installation
            #TODO: change the env variable when dir structure is changed
            export HADOOP_HOME=`dirname "$this"`/..
            export HADOOP_COMMON_HOME="$ {HADOOP_HOME}

            "

          Show
          Zhenhua (Gerald) Guo added a comment - You can add following code to files "hdfs-config.sh" and "mapred-config.sh". this="$ {BASH_SOURCE-$0} " while [ -h "$this" ]; do ls=`ls -ld "$this"` link=`expr "$ls" : '. -> (. )$'` if expr "$link" : '. /. ' > /dev/null; then this="$link" else this=`dirname "$this"`/"$link" fi done convert relative path to absolute path common_bin=`dirname "$this"` script=`basename "$this"` common_bin=`cd "$common_bin"; pwd` this="$common_bin/$script" the root of the Hadoop installation #TODO: change the env variable when dir structure is changed export HADOOP_HOME=`dirname "$this"`/.. export HADOOP_COMMON_HOME="$ {HADOOP_HOME} "
          Hide
          Zhenhua (Gerald) Guo added a comment -

          Sorry for the weird formatting in my last comment.

          this="$

          {BASH_SOURCE-$0}

          "
          while [ -h "$this" ]; do
          ls=`ls -ld "$this"`
          link=`expr "$ls" : '.-> (.)$'`
          if expr "$link" : './.' > /dev/null; then
          this="$link"
          else
          this=`dirname "$this"`/"$link"
          fi
          done

          # convert relative path to absolute path
          common_bin=`dirname "$this"`
          script=`basename "$this"`
          common_bin=`cd "$common_bin"; pwd`
          this="$common_bin/$script"

          # the root of the Hadoop installation
          #TODO: change the env variable when dir structure is changed
          export HADOOP_HOME=`dirname "$this"`/..
          export HADOOP_COMMON_HOME="$

          {HADOOP_HOME}

          "

          Show
          Zhenhua (Gerald) Guo added a comment - Sorry for the weird formatting in my last comment. this="$ {BASH_SOURCE-$0} " while [ -h "$this" ]; do ls=`ls -ld "$this"` link=`expr "$ls" : '. -> (. )$'` if expr "$link" : '. /. ' > /dev/null; then this="$link" else this=`dirname "$this"`/"$link" fi done # convert relative path to absolute path common_bin=`dirname "$this"` script=`basename "$this"` common_bin=`cd "$common_bin"; pwd` this="$common_bin/$script" # the root of the Hadoop installation #TODO: change the env variable when dir structure is changed export HADOOP_HOME=`dirname "$this"`/.. export HADOOP_COMMON_HOME="$ {HADOOP_HOME} "
          Hide
          Nigel Daley added a comment -

          From Tom: this is a regression from 0.20

          Show
          Nigel Daley added a comment - From Tom: this is a regression from 0.20
          Hide
          thron_xv added a comment -

          To avoid this issue, you can configure HADOOP_HOME in environment.

          For example in Cygwin, just do as below:
          export HADOOP_HOME=/cygdrive/c/hadoop/

          Show
          thron_xv added a comment - To avoid this issue, you can configure HADOOP_HOME in environment. For example in Cygwin, just do as below: export HADOOP_HOME=/cygdrive/c/hadoop/
          Hide
          Nigel Daley added a comment -

          Tom, any update on this for 0.22?

          Show
          Nigel Daley added a comment - Tom, any update on this for 0.22?
          Hide
          Tom White added a comment -

          Yes, please see the linked JIRAs HDFS-1823 and MAPREDUCE-2428 for the patches. There is no common part, so this JIRA can be closed when the linked JIRAs are.

          Show
          Tom White added a comment - Yes, please see the linked JIRAs HDFS-1823 and MAPREDUCE-2428 for the patches. There is no common part, so this JIRA can be closed when the linked JIRAs are.

            People

            • Assignee:
              Tom White
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development