Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5621

mr-jobhistory-daemon.sh doesn't have to execute mkdir and chown all the time

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      mr-jobhistory-daemon.sh executes mkdir and chown command to output the log files.
      This is always executed with or without a directory. In addition, this is executed not only starting daemon but also stopping daemon.
      It add "if" like hadoop-daemon.sh and yarn-daemon.sh and should control it.

      1. MAPREDUCE-5621.patch
        0.7 kB
        Shinichi Yamashita
      2. MAPREDUCE-5621-branch-2.02.patch
        0.7 kB
        Junping Du
      3. MAPREDUCE-5621-branch-2.patch
        0.7 kB
        Junping Du

        Issue Links

          Activity

          Hide
          sinchii Shinichi Yamashita added a comment -

          I attach a patch file.

          Show
          sinchii Shinichi Yamashita added a comment - I attach a patch file.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12613541/MAPREDUCE-5621.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4195//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4195//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12613541/MAPREDUCE-5621.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4195//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4195//console This message is automatically generated.
          Hide
          jwang302 Jinghui Wang added a comment -

          +1. Since if user has write access to the log directory, there is no point to create it again. This can potential cause problems if $MAPRED_LOG_DIR is owned by someone other than the user running the mr-jobhistory-daemon.sh.

          Show
          jwang302 Jinghui Wang added a comment - +1. Since if user has write access to the log directory, there is no point to create it again. This can potential cause problems if $MAPRED_LOG_DIR is owned by someone other than the user running the mr-jobhistory-daemon.sh.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12613541/MAPREDUCE-5621.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4568//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4568//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12613541/MAPREDUCE-5621.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4568//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4568//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12613541/MAPREDUCE-5621.patch
          Optional Tests shellcheck
          git revision trunk / f1a152c
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5542/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12613541/MAPREDUCE-5621.patch Optional Tests shellcheck git revision trunk / f1a152c Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5542/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12613541/MAPREDUCE-5621.patch
          Optional Tests shellcheck
          git revision trunk / f1a152c
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5544/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12613541/MAPREDUCE-5621.patch Optional Tests shellcheck git revision trunk / f1a152c Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5544/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Reopen this ticket as the issue still exists on branch-2.

          Show
          djp Junping Du added a comment - Reopen this ticket as the issue still exists on branch-2.
          Hide
          djp Junping Du added a comment -

          Rename the patch for branch-2. The latest patch LGTM. Will commit it 24 hours if no further comments.

          Show
          djp Junping Du added a comment - Rename the patch for branch-2. The latest patch LGTM. Will commit it 24 hours if no further comments.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 33s Docker mode activated.
          0 shelldocs 0m 14s Shelldocs was not available.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          -1 mvninstall 8m 35s root in branch-2 failed.
          +1 mvnsite 2m 13s branch-2 passed
          +1 mvnsite 2m 6s the patch passed
          -1 shellcheck 0m 11s The patch generated 2 new + 502 unchanged - 2 fixed = 504 total (was 504)
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 unit 1m 39s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_131.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          17m 46s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5e40efe
          JIRA Issue MAPREDUCE-5621
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876672/MAPREDUCE-5621-branch-2.patch
          Optional Tests asflicense mvnsite unit shellcheck shelldocs
          uname Linux 50f5bee35505 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / e494a16
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7022/artifact/patchprocess/branch-mvninstall-root.txt
          shellcheck v0.4.6
          shellcheck https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7022/artifact/patchprocess/diff-patch-shellcheck.txt
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7022/testReport/
          modules C: hadoop-mapreduce-project U: hadoop-mapreduce-project
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7022/console
          Powered by Apache Yetus 0.4.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 33s Docker mode activated. 0 shelldocs 0m 14s Shelldocs was not available. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 mvninstall 8m 35s root in branch-2 failed. +1 mvnsite 2m 13s branch-2 passed +1 mvnsite 2m 6s the patch passed -1 shellcheck 0m 11s The patch generated 2 new + 502 unchanged - 2 fixed = 504 total (was 504) +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 1m 39s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_131. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 17m 46s Subsystem Report/Notes Docker Image:yetus/hadoop:5e40efe JIRA Issue MAPREDUCE-5621 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876672/MAPREDUCE-5621-branch-2.patch Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux 50f5bee35505 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / e494a16 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7022/artifact/patchprocess/branch-mvninstall-root.txt shellcheck v0.4.6 shellcheck https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7022/artifact/patchprocess/diff-patch-shellcheck.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7022/testReport/ modules C: hadoop-mapreduce-project U: hadoop-mapreduce-project Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7022/console Powered by Apache Yetus 0.4.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 16m 37s Docker mode activated.
          0 shelldocs 0m 12s Shelldocs was not available.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          -1 mvninstall 9m 16s root in branch-2 failed.
          +1 mvnsite 1m 45s branch-2 passed
          +1 mvnsite 1m 39s the patch passed
          -1 shellcheck 0m 8s The patch generated 2 new + 502 unchanged - 2 fixed = 504 total (was 504)
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 unit 1m 18s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_131.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          32m 28s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5e40efe
          JIRA Issue MAPREDUCE-5621
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876672/MAPREDUCE-5621-branch-2.patch
          Optional Tests asflicense mvnsite unit shellcheck shelldocs
          uname Linux cb67c9726cf6 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / e494a16
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7023/artifact/patchprocess/branch-mvninstall-root.txt
          shellcheck v0.4.6
          shellcheck https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7023/artifact/patchprocess/diff-patch-shellcheck.txt
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7023/testReport/
          modules C: hadoop-mapreduce-project U: hadoop-mapreduce-project
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7023/console
          Powered by Apache Yetus 0.4.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 37s Docker mode activated. 0 shelldocs 0m 12s Shelldocs was not available. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 mvninstall 9m 16s root in branch-2 failed. +1 mvnsite 1m 45s branch-2 passed +1 mvnsite 1m 39s the patch passed -1 shellcheck 0m 8s The patch generated 2 new + 502 unchanged - 2 fixed = 504 total (was 504) +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 1m 18s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_131. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 32m 28s Subsystem Report/Notes Docker Image:yetus/hadoop:5e40efe JIRA Issue MAPREDUCE-5621 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876672/MAPREDUCE-5621-branch-2.patch Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux cb67c9726cf6 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / e494a16 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7023/artifact/patchprocess/branch-mvninstall-root.txt shellcheck v0.4.6 shellcheck https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7023/artifact/patchprocess/diff-patch-shellcheck.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7023/testReport/ modules C: hadoop-mapreduce-project U: hadoop-mapreduce-project Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7023/console Powered by Apache Yetus 0.4.0 http://yetus.apache.org This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Fix the warning issue in 02 patch.

          Show
          djp Junping Du added a comment - Fix the warning issue in 02 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          0 shelldocs 0m 7s Shelldocs was not available.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          -1 mvninstall 6m 37s root in branch-2 failed.
          +1 mvnsite 1m 45s branch-2 passed
          +1 mvnsite 1m 42s the patch passed
          +1 shellcheck 0m 8s The patch generated 0 new + 502 unchanged - 2 fixed = 502 total (was 504)
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 unit 1m 20s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_131.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          13m 34s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:5e40efe
          JIRA Issue MAPREDUCE-5621
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876729/MAPREDUCE-5621-branch-2.02.patch
          Optional Tests asflicense mvnsite unit shellcheck shelldocs
          uname Linux 7f8f5541bcbd 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 41e83b2
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7024/artifact/patchprocess/branch-mvninstall-root.txt
          shellcheck v0.4.6
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7024/testReport/
          modules C: hadoop-mapreduce-project U: hadoop-mapreduce-project
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7024/console
          Powered by Apache Yetus 0.4.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. 0 shelldocs 0m 7s Shelldocs was not available. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 mvninstall 6m 37s root in branch-2 failed. +1 mvnsite 1m 45s branch-2 passed +1 mvnsite 1m 42s the patch passed +1 shellcheck 0m 8s The patch generated 0 new + 502 unchanged - 2 fixed = 502 total (was 504) +1 whitespace 0m 0s The patch has no whitespace issues. +1 unit 1m 20s hadoop-mapreduce-project in the patch passed with JDK v1.7.0_131. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 13m 34s Subsystem Report/Notes Docker Image:yetus/hadoop:5e40efe JIRA Issue MAPREDUCE-5621 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876729/MAPREDUCE-5621-branch-2.02.patch Optional Tests asflicense mvnsite unit shellcheck shelldocs uname Linux 7f8f5541bcbd 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 41e83b2 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7024/artifact/patchprocess/branch-mvninstall-root.txt shellcheck v0.4.6 JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7024/testReport/ modules C: hadoop-mapreduce-project U: hadoop-mapreduce-project Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7024/console Powered by Apache Yetus 0.4.0 http://yetus.apache.org This message was automatically generated.
          Hide
          jianhe Jian He added a comment -

          lgtm

          Show
          jianhe Jian He added a comment - lgtm
          Hide
          djp Junping Du added a comment -

          Thanks Jian He for additional review. I have commit the patch to branch-2, branch-2.8 and branch-2.8.2. Thanks Shinichi Yamashita for patch contribution!

          Show
          djp Junping Du added a comment - Thanks Jian He for additional review. I have commit the patch to branch-2, branch-2.8 and branch-2.8.2. Thanks Shinichi Yamashita for patch contribution!

            People

            • Assignee:
              sinchii Shinichi Yamashita
              Reporter:
              sinchii Shinichi Yamashita
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development