Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4581

DataNode#checkDiskError should not be called on network errors

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.1, 3.0.0, 0.23.7, 2.0.4-alpha
    • Fix Version/s: 0.23.7, 2.0.4-alpha, 1.2.1
    • Component/s: datanode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    1. HDFS-4581-branch1.patch
      0.8 kB
      Rohit Kochar
    2. HDFS-4581.patch
      3 kB
      Rohit Kochar
    3. HDFS-4581-branch1.patch
      2 kB
      Rohit Kochar
    4. HDFS-4581.patch
      3 kB
      Rohit Kochar
    5. HDFS-4581-branch1.patch
      2 kB
      Rohit Kochar
    6. HDFS-4581.patch
      3 kB
      Rohit Kochar
    7. HDFS-4581.patch
      2 kB
      Rohit Kochar
    8. HDFS-4581-branch1.patch
      2 kB
      Rohit Kochar
    9. HDFS-4581.patch
      2 kB
      Rohit Kochar

      Issue Links

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        6d 8h 58m 1 Rohit Kochar 17/Mar/13 13:57
        Patch Available Patch Available Resolved Resolved
        10d 56m 1 Kihwal Lee 27/Mar/13 14:53
        Resolved Resolved Closed Closed
        29d 11h 20m 1 Arun C Murthy 26/Apr/13 03:14
        Karen Clark made changes -
        Link This issue is duplicated by HDFS-2095 [ HDFS-2095 ]
        Suresh Srinivas made changes -
        Fix Version/s 1.2.1 [ 12324148 ]
        Fix Version/s 1.3.0 [ 12324328 ]
        Target Version/s 1.2.1 [ 12324148 ]
        Hide
        Suresh Srinivas added a comment -

        I merged this change to branch 1.2 to be picked up for 1.2.1

        Show
        Suresh Srinivas added a comment - I merged this change to branch 1.2 to be picked up for 1.2.1
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Chris Nauroth added a comment -

        In HDFS-4699, I have added more cases to the network error filtering logic, based on reviewing logs from tests of rapid NN failovers.

        Show
        Chris Nauroth added a comment - In HDFS-4699 , I have added more cases to the network error filtering logic, based on reviewing logs from tests of rapid NN failovers.
        Chris Nauroth made changes -
        Link This issue is related to HDFS-4699 [ HDFS-4699 ]
        Harsh J made changes -
        Target Version/s 1.2.0, 3.0.0 [ 12321657, 12320356 ]
        Harsh J made changes -
        Fix Version/s 1.3.0 [ 12324328 ]
        Fix Version/s 3.0.0 [ 12320356 ]
        Hide
        Harsh J added a comment -

        Applying Fix Version 1.3.0 for the branch-1 commit done here; seems to have been missed post commit.

        Show
        Harsh J added a comment - Applying Fix Version 1.3.0 for the branch-1 commit done here; seems to have been missed post commit.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1386 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1386/)
        HDFS-4581. DataNode.checkDiskError should not be called on network errors. Contributed by Rohit Kochar. (Revision 1461597)

        Result = SUCCESS
        kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461597
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1386 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1386/ ) HDFS-4581 . DataNode.checkDiskError should not be called on network errors. Contributed by Rohit Kochar. (Revision 1461597) Result = SUCCESS kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461597 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1358 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1358/)
        HDFS-4581. DataNode.checkDiskError should not be called on network errors. Contributed by Rohit Kochar. (Revision 1461597)

        Result = FAILURE
        kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461597
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1358 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1358/ ) HDFS-4581 . DataNode.checkDiskError should not be called on network errors. Contributed by Rohit Kochar. (Revision 1461597) Result = FAILURE kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461597 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #567 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/567/)
        svn merge -c 1461597 Merging from trunk to branch-0.23 to fix HDFS-4581. (Revision 1461604)

        Result = SUCCESS
        kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461604
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #567 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/567/ ) svn merge -c 1461597 Merging from trunk to branch-0.23 to fix HDFS-4581 . (Revision 1461604) Result = SUCCESS kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461604 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #169 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/169/)
        HDFS-4581. DataNode.checkDiskError should not be called on network errors. Contributed by Rohit Kochar. (Revision 1461597)

        Result = FAILURE
        kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461597
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #169 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/169/ ) HDFS-4581 . DataNode.checkDiskError should not be called on network errors. Contributed by Rohit Kochar. (Revision 1461597) Result = FAILURE kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461597 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Kihwal Lee made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Hide
        Kihwal Lee added a comment -

        Matt Foley If you want it to be in any of branch-1.x, please pull in and set the fix version. I couldn't add 1.3.0 to fix version since it does not exist in jira yet.

        Show
        Kihwal Lee added a comment - Matt Foley If you want it to be in any of branch-1.x, please pull in and set the fix version. I couldn't add 1.3.0 to fix version since it does not exist in jira yet.
        Kihwal Lee made changes -
        Fix Version/s 3.0.0 [ 12320356 ]
        Fix Version/s 0.23.7 [ 12323955 ]
        Fix Version/s 2.0.4-alpha [ 12324136 ]
        Hide
        Kihwal Lee added a comment -

        I've committed this to branch-1, branch-0.23, branch-2 and trunk. Thanks for reporting the issue and contributing the patches, Rohit.

        Show
        Kihwal Lee added a comment - I've committed this to branch-1, branch-0.23, branch-2 and trunk. Thanks for reporting the issue and contributing the patches, Rohit.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3534 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3534/)
        HDFS-4581. DataNode.checkDiskError should not be called on network errors. Contributed by Rohit Kochar. (Revision 1461597)

        Result = SUCCESS
        kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461597
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3534 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3534/ ) HDFS-4581 . DataNode.checkDiskError should not be called on network errors. Contributed by Rohit Kochar. (Revision 1461597) Result = SUCCESS kihwal : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461597 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Kihwal Lee made changes -
        Assignee Rohit Kochar [ rohit_kochar ]
        Hide
        Kihwal Lee added a comment -

        I ran test-patch. The findbugs report is incorrect. I reran and it had exactly same number of warnings before and after the patch.

        [exec] -1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
        [exec] Please justify why no tests are needed for this patch.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] -1 findbugs. The patch appears to introduce 416 new Findbugs (version 1.3.9) warnings.

        +1 Everything looks okay.

        Show
        Kihwal Lee added a comment - I ran test-patch. The findbugs report is incorrect. I reran and it had exactly same number of warnings before and after the patch. [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 416 new Findbugs (version 1.3.9) warnings. +1 Everything looks okay.
        Hide
        Kihwal Lee added a comment -

        Rohit, branch-1 precommit does not run automatically. You can run it manually by building for "test-patch" target.

        Show
        Kihwal Lee added a comment - Rohit, branch-1 precommit does not run automatically. You can run it manually by building for "test-patch" target.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574388/HDFS-4581.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4117//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4117//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574388/HDFS-4581.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4117//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4117//console This message is automatically generated.
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12574388 ]
        Rohit Kochar made changes -
        Attachment HDFS-4581-branch1.patch [ 12574387 ]
        Kihwal Lee made changes -
        Link This issue is related to HDFS-4544 [ HDFS-4544 ]
        Kihwal Lee made changes -
        Fix Version/s 1.2.0 [ 12321657 ]
        Affects Version/s 0.23.7 [ 12323955 ]
        Target Version/s 1.2.0 [ 12321657 ] 1.2.0, 3.0.0 [ 12321657, 12320356 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574355/HDFS-4581.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4115//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4115//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574355/HDFS-4581.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4115//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4115//console This message is automatically generated.
        Hide
        Kihwal Lee added a comment -

        Please run test-patch for branch-1 and post the result.

        Show
        Kihwal Lee added a comment - Please run test-patch for branch-1 and post the result.
        Kihwal Lee made changes -
        Affects Version/s 3.0.0 [ 12320356 ]
        Affects Version/s 2.0.4-alpha [ 12324136 ]
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12574355 ]
        Hide
        Rohit Kochar added a comment -

        Thanks Kihwal Lee.Taken care of it.

        Show
        Rohit Kochar added a comment - Thanks Kihwal Lee.Taken care of it.
        Hide
        Kihwal Lee added a comment -

        I think we don't need the extra imports in BlockReceiver.java. (trunk version)

        Show
        Kihwal Lee added a comment - I think we don't need the extra imports in BlockReceiver.java. (trunk version)
        Hide
        Amareshwari Sriramadasu added a comment -

        +1

        Show
        Amareshwari Sriramadasu added a comment - +1
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574175/HDFS-4581.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4112//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4112//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574175/HDFS-4581.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4112//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4112//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574171/HDFS-4581.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4111//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4111//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574171/HDFS-4581.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4111//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4111//console This message is automatically generated.
        Hide
        Rohit Kochar added a comment -

        We don't need a test case here because we are just skipping the disk check in case of network exceptions.

        Show
        Rohit Kochar added a comment - We don't need a test case here because we are just skipping the disk check in case of network exceptions.
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12574175 ]
        Rohit Kochar made changes -
        Attachment HDFS-4581-branch1.patch [ 12574174 ]
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12574171 ]
        Hide
        Rohit Kochar added a comment -

        Thanks Amareshwari for your valuable inputs.
        As per you suggestion i have moved the check for the type of exception to the method checkDiskError() instead of checking it on each call of this method.
        I have also fixed the indentation issue that you pointed.
        Thanks once again.

        Show
        Rohit Kochar added a comment - Thanks Amareshwari for your valuable inputs. As per you suggestion i have moved the check for the type of exception to the method checkDiskError() instead of checking it on each call of this method. I have also fixed the indentation issue that you pointed. Thanks once again.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574170/HDFS-4581-branch1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4110//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574170/HDFS-4581-branch1.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4110//console This message is automatically generated.
        Rohit Kochar made changes -
        Attachment HDFS-4581-branch1.patch [ 12574170 ]
        Hide
        Amareshwari Sriramadasu added a comment -

        Some comments :

        1. Please take care of 80 character line indetation.
        2. Can you add java.nio.channels.ClosedByInterruptException also to the check?
        3. Can you add the check in checkDiskError(Exception) method sothat we need not change in all callers? Also, you can change callers to call checkDiskError(Exception) instead of directly calling checkDiskError(), if it because of exception and not error.
        Show
        Amareshwari Sriramadasu added a comment - Some comments : Please take care of 80 character line indetation. Can you add java.nio.channels.ClosedByInterruptException also to the check? Can you add the check in checkDiskError(Exception) method sothat we need not change in all callers? Also, you can change callers to call checkDiskError(Exception) instead of directly calling checkDiskError(), if it because of exception and not error.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574076/HDFS-4581.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4108//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4108//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574076/HDFS-4581.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4108//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4108//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574076/HDFS-4581.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4107//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4107//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574076/HDFS-4581.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4107//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4107//console This message is automatically generated.
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12574076 ]
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12574070 ]
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12574065 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574070/HDFS-4581.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4106//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574070/HDFS-4581.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4106//console This message is automatically generated.
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12574070 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574065/HDFS-4581.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4105//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574065/HDFS-4581.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4105//console This message is automatically generated.
        Rohit Kochar made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12574065 ]
        Hide
        Rohit Kochar added a comment -

        adding check for broken pipe as well

        Show
        Rohit Kochar added a comment - adding check for broken pipe as well
        Rohit Kochar made changes -
        Attachment HDFS-4581.patch [ 12573022 ]
        Rohit Kochar made changes -
        Attachment HDFS-4581-branch1.patch [ 12574064 ]
        Hide
        Amareshwari Sriramadasu added a comment -

        Seeing this also,

        2013-03-15 05:45:01,688 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: checkDiskError: exception:
        java.nio.channels.ClosedByInterruptException
                at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
                at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:341)
                at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55)
                at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
                at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146)
                at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107)
                at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
                at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
                at java.io.DataOutputStream.flush(DataOutputStream.java:106)
                at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.lastDataNodeRun(BlockReceiver.java:836)
                at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:913)
                at java.lang.Thread.run(Thread.java:662)
        
        Show
        Amareshwari Sriramadasu added a comment - Seeing this also, 2013-03-15 05:45:01,688 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: checkDiskError: exception: java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:341) at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123) at java.io.DataOutputStream.flush(DataOutputStream.java:106) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.lastDataNodeRun(BlockReceiver.java:836) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:913) at java.lang.Thread.run(Thread.java:662)
        Hide
        Amareshwari Sriramadasu added a comment -

        The other network error still not covered is 'java.io.IOException: Broken pipe' as seen at https://issues.apache.org/jira/browse/HDFS-2095?focusedCommentId=13093793&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13093793. Shall we check for exception messages with 'Broken pipe' and skip that also?

        Other than that patch looks good to me.

        Show
        Amareshwari Sriramadasu added a comment - The other network error still not covered is 'java.io.IOException: Broken pipe' as seen at https://issues.apache.org/jira/browse/HDFS-2095?focusedCommentId=13093793&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13093793 . Shall we check for exception messages with 'Broken pipe' and skip that also? Other than that patch looks good to me.
        Amareshwari Sriramadasu made changes -
        Component/s datanode [ 12312927 ]
        Amareshwari Sriramadasu made changes -
        Fix Version/s 1.2.0 [ 12321657 ]
        Affects Version/s 1.1.1 [ 12321656 ]
        Target Version/s 1.2.0 [ 12321657 ]
        Rohit Kochar made changes -
        Field Original Value New Value
        Attachment HDFS-4581.patch [ 12573022 ]
        Hide
        Rohit Kochar added a comment -

        This patch avoids running checkdisk when a socket timeout occurs or any other socket exception is thrown.

        Show
        Rohit Kochar added a comment - This patch avoids running checkdisk when a socket timeout occurs or any other socket exception is thrown.
        Hide
        Rohit Kochar added a comment -

        We are facing an issue in which most of checkDiskError() invocations are happening due to network related exceptions in BlockReciever$PacketResponder.run() and DataNode$DataTransfer.run().
        Since the current code has a common catch clause for all types of exceptions hence even for network related IO Exceptions chedkDiskError() in executed which lead to check storm thereby slowing the datanode.

        Show
        Rohit Kochar added a comment - We are facing an issue in which most of checkDiskError() invocations are happening due to network related exceptions in BlockReciever$PacketResponder.run() and DataNode$DataTransfer.run(). Since the current code has a common catch clause for all types of exceptions hence even for network related IO Exceptions chedkDiskError() in executed which lead to check storm thereby slowing the datanode.
        Rohit Kochar created issue -

          People

          • Assignee:
            Rohit Kochar
            Reporter:
            Rohit Kochar
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development