Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6587

Remove unused params in connection-related methods of Fetcher

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There are some unused params in Fecther#openConnectionWithRetry.The code is following:

      private void openConnectionWithRetry(MapHost host,
            Set<TaskAttemptID> remaining, URL url) throws IOException {
          long startTime = Time.monotonicNow();
          boolean shouldWait = true;
          while (shouldWait) {
            try {
              openConnection(url);
              shouldWait = false;
            } catch (IOException e) {
              if (!fetchRetryEnabled) {
                 // throw exception directly if fetch's retry is not enabled
                 throw e;
              }
              if ((Time.monotonicNow() - startTime) >= this.fetchRetryTimeout) {
                LOG.warn("Failed to connect to host: " + url + "after " 
                    + fetchRetryTimeout + " milliseconds.");
                throw e;
              }
              try {
                Thread.sleep(this.fetchRetryInterval);
              } catch (InterruptedException e1) {
                if (stopped) {
                  return;
                }
              }
            }
          }
        }
      

      we can see that the param remaing and host is not be used in this method. So we need to remove these param and update the method params which invoke this method.

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10324 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10324/)
        MAPREDUCE-6587. Remove unused params in connection-related methods of (aajisaka: rev 8cc4a67059e37b2083cd5468b35a64a403a3e3ae)

        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10324 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10324/ ) MAPREDUCE-6587 . Remove unused params in connection-related methods of (aajisaka: rev 8cc4a67059e37b2083cd5468b35a64a403a3e3ae) (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 40s trunk passed
        +1 compile 0m 26s trunk passed
        +1 checkstyle 0m 18s trunk passed
        +1 mvnsite 0m 28s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 47s trunk passed
        +1 javadoc 0m 22s trunk passed
        +1 mvninstall 0m 24s the patch passed
        +1 compile 0m 22s the patch passed
        +1 javac 0m 22s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 0m 58s the patch passed
        +1 javadoc 0m 19s the patch passed
        +1 unit 2m 6s hadoop-mapreduce-client-core in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        16m 27s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779187/MAPREDUCE-6587.001.patch
        JIRA Issue MAPREDUCE-6587
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux d0d190161c03 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c49333b
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6690/testReport/
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6690/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 40s trunk passed +1 compile 0m 26s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 28s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 47s trunk passed +1 javadoc 0m 22s trunk passed +1 mvninstall 0m 24s the patch passed +1 compile 0m 22s the patch passed +1 javac 0m 22s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 58s the patch passed +1 javadoc 0m 19s the patch passed +1 unit 2m 6s hadoop-mapreduce-client-core in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 16m 27s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779187/MAPREDUCE-6587.001.patch JIRA Issue MAPREDUCE-6587 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d0d190161c03 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c49333b Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6690/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6690/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Committed this to trunk, branch-2, and branch-2.8. Thanks Yiqun Lin for cleaning up the code.

        Show
        ajisakaa Akira Ajisaka added a comment - Committed this to trunk, branch-2, and branch-2.8. Thanks Yiqun Lin for cleaning up the code.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        LGTM, +1.

        Show
        ajisakaa Akira Ajisaka added a comment - LGTM, +1.
        Hide
        linyiqun Yiqun Lin added a comment -

        Allen Wittenauer, could you have time review patch? This is a obvious problem.

        Show
        linyiqun Yiqun Lin added a comment - Allen Wittenauer , could you have time review patch? This is a obvious problem.
        Hide
        linyiqun Yiqun Lin added a comment -

        Have someone review my patch?

        Show
        linyiqun Yiqun Lin added a comment - Have someone review my patch?
        Hide
        linyiqun Yiqun Lin added a comment -

        The jenkins findbug-warnings and asflicense-warning likely not related.

        Show
        linyiqun Yiqun Lin added a comment - The jenkins findbug-warnings and asflicense-warning likely not related.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 9m 51s trunk passed
        +1 compile 0m 37s trunk passed with JDK v1.8.0_66
        +1 compile 0m 33s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 41s trunk passed
        +1 mvneclipse 0m 16s trunk passed
        -1 findbugs 1m 32s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings.
        +1 javadoc 0m 34s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 33s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 36s the patch passed
        +1 compile 0m 37s the patch passed with JDK v1.8.0_66
        +1 javac 0m 37s the patch passed
        +1 compile 0m 30s the patch passed with JDK v1.7.0_91
        +1 javac 0m 30s the patch passed
        +1 checkstyle 0m 16s the patch passed
        +1 mvnsite 0m 41s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 43s the patch passed
        +1 javadoc 0m 37s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 34s the patch passed with JDK v1.7.0_91
        +1 unit 2m 53s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_66.
        +1 unit 2m 48s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_91.
        -1 asflicense 0m 27s Patch generated 1 ASF License warnings.
        28m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779187/MAPREDUCE-6587.001.patch
        JIRA Issue MAPREDUCE-6587
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 130afd274004 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8c180a1
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6227/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6227/testReport/
        asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6227/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6227/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 51s trunk passed +1 compile 0m 37s trunk passed with JDK v1.8.0_66 +1 compile 0m 33s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 41s trunk passed +1 mvneclipse 0m 16s trunk passed -1 findbugs 1m 32s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 2 extant Findbugs warnings. +1 javadoc 0m 34s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 33s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 36s the patch passed +1 compile 0m 37s the patch passed with JDK v1.8.0_66 +1 javac 0m 37s the patch passed +1 compile 0m 30s the patch passed with JDK v1.7.0_91 +1 javac 0m 30s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 41s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 43s the patch passed +1 javadoc 0m 37s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 34s the patch passed with JDK v1.7.0_91 +1 unit 2m 53s hadoop-mapreduce-client-core in the patch passed with JDK v1.8.0_66. +1 unit 2m 48s hadoop-mapreduce-client-core in the patch passed with JDK v1.7.0_91. -1 asflicense 0m 27s Patch generated 1 ASF License warnings. 28m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12779187/MAPREDUCE-6587.001.patch JIRA Issue MAPREDUCE-6587 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 130afd274004 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8c180a1 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6227/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6227/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6227/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6227/console This message was automatically generated.
        Hide
        linyiqun Yiqun Lin added a comment -

        I attach a initial patch. Thanks reviewing!

        Show
        linyiqun Yiqun Lin added a comment - I attach a initial patch. Thanks reviewing!

          People

          • Assignee:
            linyiqun Yiqun Lin
            Reporter:
            linyiqun Yiqun Lin
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development