Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha2, HADOOP-13341
    • Component/s: scripts
    • Labels:
      None

      Description

      Move HADOOP_CLIENT_OPTS handling to be in a function so that it's easier to get the proper ordering when needing to append other parts to HADOOP_OPTS prior to execution. This also greatly simplifies a lot of code all over the place.

      1. HADOOP-13355-HADOOP-13341.01.patch
        18 kB
        Allen Wittenauer
      2. HADOOP-13355-HADOOP-13341.00.patch
        17 kB
        Allen Wittenauer

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 4m 19s Maven dependency ordering for branch
        +1 mvninstall 8m 25s HADOOP-13341 passed
        +1 mvnsite 10m 31s HADOOP-13341 passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvnsite 10m 34s the patch passed
        +1 shellcheck 0m 12s There were no new shellcheck issues.
        +1 shelldocs 0m 9s There were no new shelldocs issues.
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 unit 1m 47s hadoop-common in the patch passed.
        +1 unit 0m 53s hadoop-hdfs in the patch passed.
        +1 unit 2m 31s hadoop-yarn in the patch passed.
        +1 unit 0m 26s hadoop-streaming in the patch passed.
        +1 unit 0m 22s hadoop-distcp in the patch passed.
        +1 unit 0m 24s hadoop-archive-logs in the patch passed.
        +1 unit 0m 25s hadoop-rumen in the patch passed.
        +1 unit 0m 18s hadoop-extras in the patch passed.
        +1 unit 0m 18s hadoop-sls in the patch passed.
        +1 unit 2m 7s hadoop-mapreduce-project in the patch passed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        45m 8s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HADOOP-13355
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825739/HADOOP-13355-HADOOP-13341.01.patch
        Optional Tests asflicense mvnsite unit shellcheck shelldocs
        uname Linux e6f0dee244af 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision HADOOP-13341 / cde3a00
        shellcheck v0.4.4
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10391/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn hadoop-tools/hadoop-streaming hadoop-tools/hadoop-distcp hadoop-tools/hadoop-archive-logs hadoop-tools/hadoop-rumen hadoop-tools/hadoop-extras hadoop-tools/hadoop-sls hadoop-mapreduce-project U: .
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10391/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 4m 19s Maven dependency ordering for branch +1 mvninstall 8m 25s HADOOP-13341 passed +1 mvnsite 10m 31s HADOOP-13341 passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvnsite 10m 34s the patch passed +1 shellcheck 0m 12s There were no new shellcheck issues. +1 shelldocs 0m 9s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 1m 47s hadoop-common in the patch passed. +1 unit 0m 53s hadoop-hdfs in the patch passed. +1 unit 2m 31s hadoop-yarn in the patch passed. +1 unit 0m 26s hadoop-streaming in the patch passed. +1 unit 0m 22s hadoop-distcp in the patch passed. +1 unit 0m 24s hadoop-archive-logs in the patch passed. +1 unit 0m 25s hadoop-rumen in the patch passed. +1 unit 0m 18s hadoop-extras in the patch passed. +1 unit 0m 18s hadoop-sls in the patch passed. +1 unit 2m 7s hadoop-mapreduce-project in the patch passed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 45m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-13355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825739/HADOOP-13355-HADOOP-13341.01.patch Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux e6f0dee244af 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision HADOOP-13341 / cde3a00 shellcheck v0.4.4 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10391/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn hadoop-tools/hadoop-streaming hadoop-tools/hadoop-distcp hadoop-tools/hadoop-archive-logs hadoop-tools/hadoop-rumen hadoop-tools/hadoop-extras hadoop-tools/hadoop-sls hadoop-mapreduce-project U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10391/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        aw Allen Wittenauer added a comment -

        01:

        • fix the shellcheck error
        • deal with the unique snowflake status of YARN_CLIENT_OPTS
        Show
        aw Allen Wittenauer added a comment - 01: fix the shellcheck error deal with the unique snowflake status of YARN_CLIENT_OPTS
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 12m 52s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 4m 21s Maven dependency ordering for branch
        +1 mvninstall 7m 2s HADOOP-13341 passed
        +1 mvnsite 9m 7s HADOOP-13341 passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvnsite 8m 42s the patch passed
        -1 shellcheck 0m 12s The patch generated 1 new + 75 unchanged - 0 fixed = 76 total (was 75)
        +1 shelldocs 0m 10s There were no new shelldocs issues.
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 unit 1m 55s hadoop-common in the patch passed.
        +1 unit 0m 48s hadoop-hdfs in the patch passed.
        +1 unit 2m 19s hadoop-yarn in the patch passed.
        +1 unit 0m 20s hadoop-streaming in the patch passed.
        +1 unit 0m 18s hadoop-distcp in the patch passed.
        +1 unit 0m 17s hadoop-archive-logs in the patch passed.
        +1 unit 0m 19s hadoop-rumen in the patch passed.
        +1 unit 0m 17s hadoop-extras in the patch passed.
        +1 unit 0m 20s hadoop-sls in the patch passed.
        +1 unit 2m 11s hadoop-mapreduce-project in the patch passed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        52m 38s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HADOOP-13355
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825717/HADOOP-13355-HADOOP-13341.00.patch
        Optional Tests asflicense mvnsite unit shellcheck shelldocs
        uname Linux ec49ed2ea6cc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision HADOOP-13341 / cde3a00
        shellcheck v0.4.4
        shellcheck https://builds.apache.org/job/PreCommit-HADOOP-Build/10389/artifact/patchprocess/diff-patch-shellcheck.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10389/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn hadoop-tools/hadoop-streaming hadoop-tools/hadoop-distcp hadoop-tools/hadoop-archive-logs hadoop-tools/hadoop-rumen hadoop-tools/hadoop-extras hadoop-tools/hadoop-sls hadoop-mapreduce-project U: .
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10389/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 12m 52s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 4m 21s Maven dependency ordering for branch +1 mvninstall 7m 2s HADOOP-13341 passed +1 mvnsite 9m 7s HADOOP-13341 passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvnsite 8m 42s the patch passed -1 shellcheck 0m 12s The patch generated 1 new + 75 unchanged - 0 fixed = 76 total (was 75) +1 shelldocs 0m 10s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 1m 55s hadoop-common in the patch passed. +1 unit 0m 48s hadoop-hdfs in the patch passed. +1 unit 2m 19s hadoop-yarn in the patch passed. +1 unit 0m 20s hadoop-streaming in the patch passed. +1 unit 0m 18s hadoop-distcp in the patch passed. +1 unit 0m 17s hadoop-archive-logs in the patch passed. +1 unit 0m 19s hadoop-rumen in the patch passed. +1 unit 0m 17s hadoop-extras in the patch passed. +1 unit 0m 20s hadoop-sls in the patch passed. +1 unit 2m 11s hadoop-mapreduce-project in the patch passed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 52m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-13355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825717/HADOOP-13355-HADOOP-13341.00.patch Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux ec49ed2ea6cc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision HADOOP-13341 / cde3a00 shellcheck v0.4.4 shellcheck https://builds.apache.org/job/PreCommit-HADOOP-Build/10389/artifact/patchprocess/diff-patch-shellcheck.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10389/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn hadoop-tools/hadoop-streaming hadoop-tools/hadoop-distcp hadoop-tools/hadoop-archive-logs hadoop-tools/hadoop-rumen hadoop-tools/hadoop-extras hadoop-tools/hadoop-sls hadoop-mapreduce-project U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10389/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 docker 2m 39s Docker failed to build yetus/hadoop:9560f25.



        Subsystem Report/Notes
        JIRA Issue HADOOP-13355
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825717/HADOOP-13355-HADOOP-13341.00.patch
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10388/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 2m 39s Docker failed to build yetus/hadoop:9560f25. Subsystem Report/Notes JIRA Issue HADOOP-13355 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825717/HADOOP-13355-HADOOP-13341.00.patch Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10388/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        aw Allen Wittenauer added a comment - - edited

        00

        • initial implementation

        Essentially, we only add HADOOP_CLIENT_OPTS onto HADOOP_OPTS when HADOOP_SUBCMD_DAEMONIZATION is set to false or empty.

        Show
        aw Allen Wittenauer added a comment - - edited 00 initial implementation Essentially, we only add HADOOP_CLIENT_OPTS onto HADOOP_OPTS when HADOOP_SUBCMD_DAEMONIZATION is set to false or empty.

          People

          • Assignee:
            aw Allen Wittenauer
            Reporter:
            aw Allen Wittenauer
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development