Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: scripts
    • Labels:
    • Release Note:
      Hide
      <!-- markdown -->

      | Old | New |
      |:---- |:---- |
      | HADOOP\_HDFS\_LOG\_DIR | HADOOP\_LOG\_DIR |
      | HADOOP\_HDFS\_LOGFILE | HADOOP\_LOGFILE |
      | HADOOP\_HDFS\_NICENESS | HADOOP\_NICENESS |
      | HADOOP\_HDFS\_STOP\_TIMEOUT | HADOOP\_STOP\_TIMEOUT |
      | HADOOP\_HDFS\_PID\_DIR | HADOOP\_PID\_DIR |
      | HADOOP\_HDFS\_ROOT\_LOGGER | HADOOP\_ROOT\_LOGGER |
      | HADOOP\_HDFS\_IDENT\_STRING | HADOOP\_IDENT\_STRING |
      | HADOOP\_MAPRED\_LOG\_DIR | HADOOP\_LOG\_DIR |
      | HADOOP\_MAPRED\_LOGFILE | HADOOP\_LOGFILE |
      | HADOOP\_MAPRED\_NICENESS | HADOOP\_NICENESS |
      | HADOOP\_MAPRED\_STOP\_TIMEOUT | HADOOP\_STOP\_TIMEOUT |
      | HADOOP\_MAPRED\_PID\_DIR | HADOOP\_PID\_DIR |
      | HADOOP\_MAPRED\_ROOT\_LOGGER | HADOOP\_ROOT\_LOGGER |
      | HADOOP\_MAPRED\_IDENT\_STRING | HADOOP\_IDENT\_STRING |
      | YARN\_CONF\_DIR | HADOOP\_CONF\_DIR |
      | YARN\_LOG\_DIR | HADOOP\_LOG\_DIR |
      | YARN\_LOGFILE | HADOOP\_LOGFILE |
      | YARN\_NICENESS | HADOOP\_NICENESS |
      | YARN\_STOP\_TIMEOUT | HADOOP\_STOP\_TIMEOUT |
      | YARN\_PID\_DIR | HADOOP\_PID\_DIR |
      | YARN\_ROOT\_LOGGER | HADOOP\_ROOT\_LOGGER |
      | YARN\_IDENT\_STRING | HADOOP\_IDENT\_STRING |
      | YARN\_OPTS | HADOOP\_OPTS |
      | YARN\_SLAVES | HADOOP\_SLAVES |
      | YARN\_USER\_CLASSPATH | HADOOP\_CLASSPATH |
      | YARN\_USER\_CLASSPATH\_FIRST | HADOOP\_USER\_CLASSPATH\_FIRST |
      | KMS\_CONFIG | HADOOP\_CONF\_DIR |
      | KMS\_LOG | HADOOP\_LOG\_DIR |
      Show
      <!-- markdown --> | Old | New | |:---- |:---- | | HADOOP\_HDFS\_LOG\_DIR | HADOOP\_LOG\_DIR | | HADOOP\_HDFS\_LOGFILE | HADOOP\_LOGFILE | | HADOOP\_HDFS\_NICENESS | HADOOP\_NICENESS | | HADOOP\_HDFS\_STOP\_TIMEOUT | HADOOP\_STOP\_TIMEOUT | | HADOOP\_HDFS\_PID\_DIR | HADOOP\_PID\_DIR | | HADOOP\_HDFS\_ROOT\_LOGGER | HADOOP\_ROOT\_LOGGER | | HADOOP\_HDFS\_IDENT\_STRING | HADOOP\_IDENT\_STRING | | HADOOP\_MAPRED\_LOG\_DIR | HADOOP\_LOG\_DIR | | HADOOP\_MAPRED\_LOGFILE | HADOOP\_LOGFILE | | HADOOP\_MAPRED\_NICENESS | HADOOP\_NICENESS | | HADOOP\_MAPRED\_STOP\_TIMEOUT | HADOOP\_STOP\_TIMEOUT | | HADOOP\_MAPRED\_PID\_DIR | HADOOP\_PID\_DIR | | HADOOP\_MAPRED\_ROOT\_LOGGER | HADOOP\_ROOT\_LOGGER | | HADOOP\_MAPRED\_IDENT\_STRING | HADOOP\_IDENT\_STRING | | YARN\_CONF\_DIR | HADOOP\_CONF\_DIR | | YARN\_LOG\_DIR | HADOOP\_LOG\_DIR | | YARN\_LOGFILE | HADOOP\_LOGFILE | | YARN\_NICENESS | HADOOP\_NICENESS | | YARN\_STOP\_TIMEOUT | HADOOP\_STOP\_TIMEOUT | | YARN\_PID\_DIR | HADOOP\_PID\_DIR | | YARN\_ROOT\_LOGGER | HADOOP\_ROOT\_LOGGER | | YARN\_IDENT\_STRING | HADOOP\_IDENT\_STRING | | YARN\_OPTS | HADOOP\_OPTS | | YARN\_SLAVES | HADOOP\_SLAVES | | YARN\_USER\_CLASSPATH | HADOOP\_CLASSPATH | | YARN\_USER\_CLASSPATH\_FIRST | HADOOP\_USER\_CLASSPATH\_FIRST | | KMS\_CONFIG | HADOOP\_CONF\_DIR | | KMS\_LOG | HADOOP\_LOG\_DIR |

      Description

      It is a very common shell pattern in 3.x to effectively replace sub-project specific vars with generics. We should have a function that does this replacement and provides a warning to the end user that the old shell var is deprecated. Additionally, we should use this shell function to deprecate the shell vars that are holdovers already.

      1. HADOOP-11460-00.patch
        7 kB
        Allen Wittenauer
      2. HADOOP-11460-01.patch
        18 kB
        Allen Wittenauer
      3. HADOOP-11460-02.patch
        24 kB
        Allen Wittenauer
      4. HADOOP-11460-03.patch
        26 kB
        John Smith
      5. HADOOP-11460-04.patch
        26 kB
        John Smith

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment -

          FYI: updated releasenote to reflect the fix in HADOOP-13349. HADOOP_USER_CLASSPATH was going to happen but didn't.

          Show
          aw Allen Wittenauer added a comment - FYI: updated releasenote to reflect the fix in HADOOP-13349 . HADOOP_USER_CLASSPATH was going to happen but didn't.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2046 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2046/)
          HADOOP-11460. Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c)

          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh
          • hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh
          • hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh
          • hadoop-mapreduce-project/conf/mapred-env.sh
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          • hadoop-mapreduce-project/bin/mapred-config.sh
          • hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2046 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2046/ ) HADOOP-11460 . Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c) hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh hadoop-mapreduce-project/conf/mapred-env.sh hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh hadoop-mapreduce-project/bin/mapred-config.sh hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #96 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/96/)
          HADOOP-11460. Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c)

          • hadoop-mapreduce-project/conf/mapred-env.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh
          • hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-mapreduce-project/bin/mapred-config.sh
          • hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh
          • hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #96 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/96/ ) HADOOP-11460 . Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c) hadoop-mapreduce-project/conf/mapred-env.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh hadoop-common-project/hadoop-common/CHANGES.txt hadoop-mapreduce-project/bin/mapred-config.sh hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #92 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/92/)
          HADOOP-11460. Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c)

          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-mapreduce-project/bin/mapred-config.sh
          • hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh
          • hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh
          • hadoop-mapreduce-project/conf/mapred-env.sh
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh
          • hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh
          • hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh
          • hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #92 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/92/ ) HADOOP-11460 . Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c) hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh hadoop-common-project/hadoop-common/CHANGES.txt hadoop-mapreduce-project/bin/mapred-config.sh hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh hadoop-mapreduce-project/conf/mapred-env.sh hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh hadoop-yarn-project/hadoop-yarn/bin/yarn
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #2027 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2027/)
          HADOOP-11460. Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c)

          • hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh
          • hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh
          • hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh
          • hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-mapreduce-project/conf/mapred-env.sh
          • hadoop-mapreduce-project/bin/mapred-config.sh
          • hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2027 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2027/ ) HADOOP-11460 . Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c) hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-mapreduce-project/conf/mapred-env.sh hadoop-mapreduce-project/bin/mapred-config.sh hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #829 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/829/)
          HADOOP-11460. Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c)

          • hadoop-mapreduce-project/conf/mapred-env.sh
          • hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh
          • hadoop-mapreduce-project/bin/mapred-config.sh
          • hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh
          • hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh
          • hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #829 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/829/ ) HADOOP-11460 . Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c) hadoop-mapreduce-project/conf/mapred-env.sh hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh hadoop-mapreduce-project/bin/mapred-config.sh hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh hadoop-yarn-project/hadoop-yarn/bin/yarn
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #95 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/95/)
          HADOOP-11460. Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c)

          • hadoop-mapreduce-project/conf/mapred-env.sh
          • hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh
          • hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-mapreduce-project/bin/mapred-config.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh
          • hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh
          • hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #95 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/95/ ) HADOOP-11460 . Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c) hadoop-mapreduce-project/conf/mapred-env.sh hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh hadoop-common-project/hadoop-common/CHANGES.txt hadoop-mapreduce-project/bin/mapred-config.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7011 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7011/)
          HADOOP-11460. Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c)

          • hadoop-mapreduce-project/conf/mapred-env.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh
          • hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh
          • hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-mapreduce-project/bin/mapred-config.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
          • hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh
          • hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh
          • hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7011 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7011/ ) HADOOP-11460 . Deprecate shell vars (John Smith via aw) (aw: rev 43d5caef5e63b3c96737cb642c1e8f31ec8c9a8c) hadoop-mapreduce-project/conf/mapred-env.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-yarn-project/hadoop-yarn/bin/yarn-config.sh hadoop-common-project/hadoop-kms/src/main/conf/kms-env.sh hadoop-yarn-project/hadoop-yarn/bin/stop-yarn.sh hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs-config.sh hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-mapreduce-project/bin/mapred-config.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh hadoop-common-project/hadoop-kms/src/main/libexec/kms-config.sh hadoop-yarn-project/hadoop-yarn/bin/start-yarn.sh hadoop-yarn-project/hadoop-yarn/conf/yarn-env.sh
          Hide
          aw Allen Wittenauer added a comment -

          +1 will commit this to trunk.

          Thanks!

          Show
          aw Allen Wittenauer added a comment - +1 will commit this to trunk. Thanks!
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12695575/HADOOP-11460-04.patch
          against trunk revision f2c9109.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          -1 release audit. The applied patch generated 1 release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5545//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5545//artifact/patchprocess/patchReleaseAuditProblems.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5545//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12695575/HADOOP-11460-04.patch against trunk revision f2c9109. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 release audit . The applied patch generated 1 release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5545//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5545//artifact/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5545//console This message is automatically generated.
          Hide
          pocky John Smith added a comment -

          OK, Allen Wittenauer, here is the new patch with your function.

          Show
          pocky John Smith added a comment - OK, Allen Wittenauer , here is the new patch with your function.
          Hide
          aw Allen Wittenauer added a comment -

          I think I found it. Also, I think since the old vars are deprecated, we shouldn't populate them. It will encourage folks to stop using them in new code... and given this is trunk, I think that's ok. Oh, and this uses quotes because YARN_OPT.

          function hadoop_deprecate_envvar
          {
            local oldvar=$1
            local newvar=$2
            local oldval=${!oldvar}
            local newval=${!newvar}
          
            if [[ -n "${oldval}" ]]; then
              hadoop_error "WARNING: ${oldvar} has been replaced by ${newvar}. Using value of ${oldvar}."
              # shellcheck disable=SC2086
              eval ${newvar}=\"${oldval}\"
          
              # shellcheck disable=SC2086
              newval=${oldval}
          
              # shellcheck disable=SC2086
              eval ${newvar}=\"${newval}\"
            fi
          }
          
          Show
          aw Allen Wittenauer added a comment - I think I found it. Also, I think since the old vars are deprecated, we shouldn't populate them. It will encourage folks to stop using them in new code... and given this is trunk, I think that's ok. Oh, and this uses quotes because YARN_OPT. function hadoop_deprecate_envvar { local oldvar=$1 local newvar=$2 local oldval=${!oldvar} local newval=${!newvar} if [[ -n "${oldval}" ]]; then hadoop_error "WARNING: ${oldvar} has been replaced by ${newvar}. Using value of ${oldvar}." # shellcheck disable=SC2086 eval ${newvar}=\ "${oldval}\" # shellcheck disable=SC2086 newval=${oldval} # shellcheck disable=SC2086 eval ${newvar}=\ "${newval}\" fi }
          Hide
          aw Allen Wittenauer added a comment -

          One subtle bug which we both missed. sbin/yarn-daemon.sh is somehow setting HADOOP_CONF_DIR and YARN_CONF_DIR to "YARN_CONF_DIR". I suspect it has something to do with null values not getting handled properly.

          Show
          aw Allen Wittenauer added a comment - One subtle bug which we both missed. sbin/yarn-daemon.sh is somehow setting HADOOP_CONF_DIR and YARN_CONF_DIR to "YARN_CONF_DIR". I suspect it has something to do with null values not getting handled properly.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12695177/HADOOP-11460-03.patch
          against trunk revision 5a0051f.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5528//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5528//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12695177/HADOOP-11460-03.patch against trunk revision 5a0051f. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms hadoop-hdfs-project/hadoop-hdfs. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5528//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5528//console This message is automatically generated.
          Hide
          pocky John Smith added a comment -

          Same as Allen Wittenauer's patch but now with KMS.

          Show
          pocky John Smith added a comment - Same as Allen Wittenauer 's patch but now with KMS.
          Hide
          aw Allen Wittenauer added a comment -

          Yes. As usual, feel free to update the patch.

          Show
          aw Allen Wittenauer added a comment - Yes. As usual, feel free to update the patch.
          Hide
          pocky John Smith added a comment -

          Should KMS get deprecated as well?

          Show
          pocky John Smith added a comment - Should KMS get deprecated as well?
          Hide
          aw Allen Wittenauer added a comment -

          BTW, I'm debating merging this patch and HADOOP-11485 into one patch.

          Show
          aw Allen Wittenauer added a comment - BTW, I'm debating merging this patch and HADOOP-11485 into one patch.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12693140/HADOOP-11460-02.patch
          against trunk revision e843a0a.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5429//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5429//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12693140/HADOOP-11460-02.patch against trunk revision e843a0a. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestEncryptionZonesWithKMS Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5429//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5429//console This message is automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          FYI: This also fixes a problem with mapred-env.sh... MAPRED_ROOT_LOGGER isn't the correct env var.

          Show
          aw Allen Wittenauer added a comment - FYI: This also fixes a problem with mapred-env.sh... MAPRED_ROOT_LOGGER isn't the correct env var.
          Hide
          aw Allen Wittenauer added a comment -

          -02:

          • Fix two shellcheck warnings
          • Remove deprecated vars from the env.sh files.
          Show
          aw Allen Wittenauer added a comment - -02: Fix two shellcheck warnings Remove deprecated vars from the env.sh files.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12692800/HADOOP-11460-01.patch
          against trunk revision 000ca83.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.ha.TestZKFailoverControllerStress
          org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5418//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5418//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12692800/HADOOP-11460-01.patch against trunk revision 000ca83. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.ha.TestZKFailoverControllerStress org.apache.hadoop.hdfs.server.blockmanagement.TestDatanodeManager Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5418//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5418//console This message is automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          -01:

          • Eliminate most (all?) of the usages of the deprecated env vars!
          Show
          aw Allen Wittenauer added a comment - -01: Eliminate most (all?) of the usages of the deprecated env vars!
          Hide
          aw Allen Wittenauer added a comment -

          -00:

          • initial version
          • deprecate a bunch of hdfs, mapred, and yarn vars
          Show
          aw Allen Wittenauer added a comment - -00: initial version deprecate a bunch of hdfs, mapred, and yarn vars

            People

            • Assignee:
              pocky John Smith
              Reporter:
              aw Allen Wittenauer
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development