Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9873

hadoop-env.sh got called multiple times

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: None
    • Component/s: scripts
    • Labels:
      None

      Description

      Ref. below, it can be seen hadoop-env.sh got called multiple times when running something like 'hadoop-daemon.sh start namenode'.

      [drankye@zkdev ~]$ cd $HADOOP_PREFIX
      [drankye@zkdev hadoop-3.0.0-SNAPSHOT]$ grep -r hadoop-env *
      libexec/hadoop-config.sh:if [ -e "${HADOOP_PREFIX}/conf/hadoop-env.sh" ]; then
      libexec/hadoop-config.sh:if [ -f "${HADOOP_CONF_DIR}/hadoop-env.sh" ]; then
      libexec/hadoop-config.sh:  . "${HADOOP_CONF_DIR}/hadoop-env.sh"
      sbin/hadoop-daemon.sh:if [ -f "${HADOOP_CONF_DIR}/hadoop-env.sh" ]; then
      sbin/hadoop-daemon.sh:  . "${HADOOP_CONF_DIR}/hadoop-env.sh"
      

      Considering the following lines in hadoop-env.sh

      # Command specific options appended to HADOOP_OPTS when specified
      export HADOOP_NAMENODE_OPTS="-Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS} -Dhdfs.audit.logger=${HDFS_AUDIT_LOGGER:-INFO,NullAppender} $HADOOP_NAMENODE_OPTS"
      

      It may end with some redundant result like below when called multiple times.

      HADOOP_NAMENODE_OPTS='-Dhadoop.security.logger=INFO,RFAS -Dhdfs.audit.logger=INFO,NullAppender -Dhadoop.security.logger=INFO,RFAS -Dhdfs.audit.logger=INFO,NullAppender '
      

      It's not a big issue for now however it would be better to be clean and avoid this since it can cause the final JAVA command line is very lengthy and hard to read.

      A possible fix would be to add a flag variable like HADOOP_ENV_INITED in hadoop-env.sh, and then at the beginning of it check the flag. If the flag evaluates true, then return immediately.

        Attachments

        1. HADOOP-9873.patch
          0.6 kB
          Kai Zheng

          Issue Links

            Activity

              People

              • Assignee:
                drankye Kai Zheng
                Reporter:
                drankye Kai Zheng
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: