Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0, 2.8.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      2014-02-20 12:08:45,141 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
      2014-02-20 12:08:45,142 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Adding auxiliary service httpshuffle, "mapreduce_shuffle"
      

      I'm seeing this in my NodeManager logs, even though things work fine. A WARN is being caused by some sort of mismatch between the name of the service (in terms of org.apache.hadoop.service.Service.getName()) and the name of the auxiliary service.

        Activity

        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2525 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2525/)
        MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2525 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2525/ ) MAPREDUCE-5763 . Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #586 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/586/)
        MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #586 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/586/ ) MAPREDUCE-5763 . Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #644 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/644/)
        MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #644 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/644/ ) MAPREDUCE-5763 . Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #1378 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1378/)
        MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1378 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1378/ ) MAPREDUCE-5763 . Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #655 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/655/)
        MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #655 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/655/ ) MAPREDUCE-5763 . Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2585 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2585/)
        MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2585 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2585/ ) MAPREDUCE-5763 . Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        Tsuyoshi Ozawa added a comment -

        Thank Akira for your reply. OK, removing 2.7.3 from target version.

        Show
        Tsuyoshi Ozawa added a comment - Thank Akira for your reply. OK, removing 2.7.3 from target version.
        Hide
        Akira AJISAKA added a comment -

        Thanks Tsuyoshi Ozawa for review and commit!

        I think that it can cause a bit surprise for users to change a log format in branch-2.7(especially for sysadmins). What do you think?

        Agree. Now I'm +1 for not committing this to branch-2.7.

        Show
        Akira AJISAKA added a comment - Thanks Tsuyoshi Ozawa for review and commit! I think that it can cause a bit surprise for users to change a log format in branch-2.7(especially for sysadmins). What do you think? Agree. Now I'm +1 for not committing this to branch-2.7.
        Hide
        Tsuyoshi Ozawa added a comment -

        Committed this to trunk and branch-2. Thanks Akira AJISAKA for the contribution.

        I think that it can cause a bit surprise for users to change a log format in branch-2.7(especially for sysadmins). What do you think?

        Show
        Tsuyoshi Ozawa added a comment - Committed this to trunk and branch-2. Thanks Akira AJISAKA for the contribution. I think that it can cause a bit surprise for users to change a log format in branch-2.7(especially for sysadmins). What do you think?
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8777 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8777/)
        MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8777 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8777/ ) MAPREDUCE-5763 . Warn message about httpshuffle in NM logs. Contributed (ozawa: rev 668e897538e317d96784f75b4e1f979c72225db7) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        Tsuyoshi Ozawa added a comment -

        +1, checking this in.

        Show
        Tsuyoshi Ozawa added a comment - +1, checking this in.
        Hide
        Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 3m 38s trunk passed
        +1 compile 0m 14s trunk passed with JDK v1.8.0_60
        +1 compile 0m 13s trunk passed with JDK v1.7.0_79
        +1 checkstyle 0m 9s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 14s trunk passed with JDK v1.8.0_60
        +1 javadoc 0m 15s trunk passed with JDK v1.7.0_79
        +1 mvninstall 0m 17s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.8.0_60
        +1 javac 0m 13s the patch passed
        +1 compile 0m 14s the patch passed with JDK v1.7.0_79
        +1 javac 0m 14s the patch passed
        +1 checkstyle 0m 8s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 36s the patch passed
        +1 javadoc 0m 13s the patch passed with JDK v1.8.0_60
        +1 javadoc 0m 15s the patch passed with JDK v1.7.0_79
        +1 unit 0m 20s hadoop-mapreduce-client-shuffle in the patch passed with JDK v1.8.0_60.
        +1 unit 0m 19s hadoop-mapreduce-client-shuffle in the patch passed with JDK v1.7.0_79.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        9m 50s



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771288/MAPREDUCE-5763.00.patch
        JIRA Issue MAPREDUCE-5763
        Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
        uname Linux d754fe33a5db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
        git revision trunk / 655d434
        Default Java 1.7.0_79
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
        findbugs v3.0.0
        JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6130/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle
        Max memory used 226MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6130/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 3m 38s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_60 +1 compile 0m 13s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 9s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 14s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 17s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_60 +1 javac 0m 13s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_79 +1 javac 0m 14s the patch passed +1 checkstyle 0m 8s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 36s the patch passed +1 javadoc 0m 13s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 15s the patch passed with JDK v1.7.0_79 +1 unit 0m 20s hadoop-mapreduce-client-shuffle in the patch passed with JDK v1.8.0_60. +1 unit 0m 19s hadoop-mapreduce-client-shuffle in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 9m 50s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771288/MAPREDUCE-5763.00.patch JIRA Issue MAPREDUCE-5763 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux d754fe33a5db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 655d434 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6130/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6130/console This message was automatically generated.
        Hide
        Akira AJISAKA added a comment -

        Attaching a patch to change the service name of ShuffleHandler.

        Show
        Akira AJISAKA added a comment - Attaching a patch to change the service name of ShuffleHandler.
        Hide
        Akira AJISAKA added a comment -

        Hi Naren Koneru, how is this issue going? If you don't have time to work on this issue, I'd like to take it over.

        Show
        Akira AJISAKA added a comment - Hi Naren Koneru , how is this issue going? If you don't have time to work on this issue, I'd like to take it over.

          People

          • Assignee:
            Akira AJISAKA
            Reporter:
            Sandy Ryza
          • Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development