Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11993

Add log info when connect to datanode socket address failed

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      In function BlockSeekTo, when connect faild to datanode socket address,log as follow:

      DFSClient.LOG.warn("Failed to connect to " + targetAddr + " for block"
      + ", add to deadNodes and continue. " + ex, ex);

      add block info may be more explicit.

      1. HADOOP-11993.002.patch
        1 kB
        chencan
      2. HADOOP-11993.patch
        1 kB
        chencan

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 16m 14s trunk passed
        +1 compile 0m 33s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 34s trunk passed
        +1 findbugs 1m 30s trunk passed
        +1 javadoc 0m 22s trunk passed
        +1 mvninstall 0m 33s the patch passed
        +1 compile 0m 29s the patch passed
        +1 javac 0m 29s the patch passed
        -0 checkstyle 0m 13s hadoop-hdfs-project/hadoop-hdfs-client: The patch generated 2 new + 41 unchanged - 0 fixed = 43 total (was 41)
        +1 mvnsite 0m 33s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 31s the patch passed
        +1 javadoc 0m 18s the patch passed
        +1 unit 1m 10s hadoop-hdfs-client in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        26m 17s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11993
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873444/HADOOP-11993.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 7f42078e9433 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 75043d3
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19943/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19943/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19943/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 16m 14s trunk passed +1 compile 0m 33s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 34s trunk passed +1 findbugs 1m 30s trunk passed +1 javadoc 0m 22s trunk passed +1 mvninstall 0m 33s the patch passed +1 compile 0m 29s the patch passed +1 javac 0m 29s the patch passed -0 checkstyle 0m 13s hadoop-hdfs-project/hadoop-hdfs-client: The patch generated 2 new + 41 unchanged - 0 fixed = 43 total (was 41) +1 mvnsite 0m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 31s the patch passed +1 javadoc 0m 18s the patch passed +1 unit 1m 10s hadoop-hdfs-client in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 26m 17s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11993 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873444/HADOOP-11993.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7f42078e9433 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 75043d3 Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19943/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19943/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19943/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        raviprak Ravi Prakash added a comment -

        +1 Will commit by EOD unless there are objections

        Show
        raviprak Ravi Prakash added a comment - +1 Will commit by EOD unless there are objections
        Hide
        vagarychen Chen Liang added a comment -

        Thanks chencan for the patch. Since this seems to be slf4j logger, how about considering using {} placeholders? e.g.

        DFSClient.LOG.warn("Failed to connect to " + targetAddr + " for block " + 
            targetBlock.getBlock() + ", add to deadNodes and continue. " + ex, ex);
        

        to something like

        DFSClient.LOG.warn("Failed to connect to {} for block {}, add to deadNodes and continue. ", targetAddr, targetBlock.getBlock(), ex);
        
        Show
        vagarychen Chen Liang added a comment - Thanks chencan for the patch. Since this seems to be slf4j logger, how about considering using {} placeholders? e.g. DFSClient.LOG.warn( "Failed to connect to " + targetAddr + " for block " + targetBlock.getBlock() + ", add to deadNodes and continue . " + ex, ex); to something like DFSClient.LOG.warn( "Failed to connect to {} for block {}, add to deadNodes and continue . " , targetAddr, targetBlock.getBlock(), ex);
        Hide
        candychencan chencan added a comment -

        Thanks for your suggestion, I have changed and add a new patch.

        Show
        candychencan chencan added a comment - Thanks for your suggestion, I have changed and add a new patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 13m 24s trunk passed
        +1 compile 0m 32s trunk passed
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 34s trunk passed
        +1 findbugs 1m 23s trunk passed
        +1 javadoc 0m 20s trunk passed
        +1 mvninstall 0m 32s the patch passed
        +1 compile 0m 29s the patch passed
        +1 javac 0m 29s the patch passed
        -0 checkstyle 0m 13s hadoop-hdfs-project/hadoop-hdfs-client: The patch generated 1 new + 41 unchanged - 0 fixed = 42 total (was 41)
        +1 mvnsite 0m 32s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 28s the patch passed
        +1 javadoc 0m 18s the patch passed
        +1 unit 1m 9s hadoop-hdfs-client in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        22m 53s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11993
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874343/HADOOP-11993.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b82f53de27da 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0111711
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20023/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-client.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20023/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20023/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 13m 24s trunk passed +1 compile 0m 32s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 34s trunk passed +1 findbugs 1m 23s trunk passed +1 javadoc 0m 20s trunk passed +1 mvninstall 0m 32s the patch passed +1 compile 0m 29s the patch passed +1 javac 0m 29s the patch passed -0 checkstyle 0m 13s hadoop-hdfs-project/hadoop-hdfs-client: The patch generated 1 new + 41 unchanged - 0 fixed = 42 total (was 41) +1 mvnsite 0m 32s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 28s the patch passed +1 javadoc 0m 18s the patch passed +1 unit 1m 9s hadoop-hdfs-client in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 22m 53s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11993 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874343/HADOOP-11993.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b82f53de27da 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0111711 Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/20023/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-client.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20023/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20023/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 12m 42s trunk passed
        +1 compile 0m 28s trunk passed
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 34s trunk passed
        +1 findbugs 1m 18s trunk passed
        +1 javadoc 0m 21s trunk passed
        +1 mvninstall 0m 29s the patch passed
        +1 compile 0m 28s the patch passed
        +1 javac 0m 28s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 29s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        +1 findbugs 1m 21s the patch passed
        +1 javadoc 0m 19s the patch passed
        +1 unit 1m 14s hadoop-hdfs-client in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        21m 58s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11993
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874349/HADOOP-11993.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 1caf8e66fc82 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0111711
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/20025/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20025/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20025/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 12m 42s trunk passed +1 compile 0m 28s trunk passed +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 34s trunk passed +1 findbugs 1m 18s trunk passed +1 javadoc 0m 21s trunk passed +1 mvninstall 0m 29s the patch passed +1 compile 0m 28s the patch passed +1 javac 0m 28s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 29s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 1m 21s the patch passed +1 javadoc 0m 19s the patch passed +1 unit 1m 14s hadoop-hdfs-client in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 21m 58s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11993 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874349/HADOOP-11993.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1caf8e66fc82 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0111711 Default Java 1.8.0_131 findbugs v3.1.0-RC1 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/20025/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20025/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20025/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 13m 46s trunk passed
        +1 compile 0m 33s trunk passed
        +1 checkstyle 0m 17s trunk passed
        +1 mvnsite 0m 34s trunk passed
        +1 findbugs 1m 31s trunk passed
        +1 javadoc 0m 22s trunk passed
        +1 mvninstall 0m 34s the patch passed
        +1 compile 0m 30s the patch passed
        +1 javac 0m 30s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 33s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 33s the patch passed
        +1 javadoc 0m 19s the patch passed
        +1 unit 1m 17s hadoop-hdfs-client in the patch passed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        24m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11993
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874350/HADOOP-11993.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 1fdc4d406c18 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0111711
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20026/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20026/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 13m 46s trunk passed +1 compile 0m 33s trunk passed +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 34s trunk passed +1 findbugs 1m 31s trunk passed +1 javadoc 0m 22s trunk passed +1 mvninstall 0m 34s the patch passed +1 compile 0m 30s the patch passed +1 javac 0m 30s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 33s the patch passed +1 javadoc 0m 19s the patch passed +1 unit 1m 17s hadoop-hdfs-client in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 24m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11993 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874350/HADOOP-11993.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1fdc4d406c18 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0111711 Default Java 1.8.0_131 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/20026/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client Console output https://builds.apache.org/job/PreCommit-HDFS-Build/20026/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        raviprak Ravi Prakash added a comment -

        Thanks Chen Liang for the review and suggestion. Thanks chencan for the contribution and updating the patch. +1. LGTM
        Committing to shortly.

        Show
        raviprak Ravi Prakash added a comment - Thanks Chen Liang for the review and suggestion. Thanks chencan for the contribution and updating the patch. +1. LGTM Committing to shortly.
        Hide
        raviprak Ravi Prakash added a comment -

        Committed to trunk and cherry-picked into branch-2.

        Show
        raviprak Ravi Prakash added a comment - Committed to trunk and cherry-picked into branch-2.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11926 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11926/)
        HDFS-11993. Add log info when connect to datanode socket address failed. (raviprak: rev a9d3412b4ce40f5ab5a18756ede7e0606b653171)

        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11926 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11926/ ) HDFS-11993 . Add log info when connect to datanode socket address failed. (raviprak: rev a9d3412b4ce40f5ab5a18756ede7e0606b653171) (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java

          People

          • Assignee:
            candychencan chencan
            Reporter:
            candychencan chencan
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development