Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3384

DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Scenraio:
      =========
      write a file
      corrupt block manually
      call append..

      2012-04-19 09:33:10,776 INFO  hdfs.DFSClient (DFSOutputStream.java:createBlockOutputStream(1059)) - Exception in createBlockOutputStream
      java.io.EOFException: Premature EOF: no length prefix available
      	at org.apache.hadoop.hdfs.protocol.HdfsProtoUtil.vintPrefixed(HdfsProtoUtil.java:162)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1039)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:939)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
      2012-04-19 09:33:10,807 WARN  hdfs.DFSClient (DFSOutputStream.java:run(549)) - DataStreamer Exception
      java.lang.NullPointerException
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:510)
      2012-04-19 09:33:10,807 WARN  hdfs.DFSClient (DFSOutputStream.java:hflush(1511)) - Error while syncing
      java.io.IOException: All datanodes 10.18.40.20:50010 are bad. Aborting...
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:908)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
      java.io.IOException: All datanodes 10.18.40.20:50010 are bad. Aborting...
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:908)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
      
      1. HDFS-3384_2.patch
        2 kB
        amith
      2. HDFS-3384_2.patch
        5 kB
        amith
      3. HDFS-3384_2.patch
        5 kB
        amith
      4. HDFS-3384.patch
        0.8 kB
        amith
      5. HDFS-3384-3.patch
        0.8 kB
        Uma Maheswara Rao G
      6. HDFS-3384-4.patch
        2 kB
        Uma Maheswara Rao G

        Activity

        Hide
        umamaheswararao Uma Maheswara Rao G added a comment -

        Amith, could please update the patch?

        Show
        umamaheswararao Uma Maheswara Rao G added a comment - Amith, could please update the patch?
        Hide
        amithdk amith added a comment -

        Yes I will provide the patch prepared by EOD

        Show
        amithdk amith added a comment - Yes I will provide the patch prepared by EOD
        Hide
        amithdk amith added a comment -

        When a pipeline setup for append/recovery may fail. If the setup fails we are setting streamerClosed but there is no condition which cause the streamer thread to stop.

        currently in the given stack trace

        java.lang.NullPointerException
        	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:510)
        

        caused has a side effect of not closing the streamer thread even after setting the streamerClosed.

        Manually verified the scenario
        write a file
        corrupt block manually
        call append

        Show
        amithdk amith added a comment - When a pipeline setup for append/recovery may fail. If the setup fails we are setting streamerClosed but there is no condition which cause the streamer thread to stop. currently in the given stack trace java.lang.NullPointerException at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:510) caused has a side effect of not closing the streamer thread even after setting the streamerClosed. Manually verified the scenario write a file corrupt block manually call append
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526178/HDFS-3384.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2396//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2396//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526178/HDFS-3384.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2396//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2396//console This message is automatically generated.
        Hide
        amithdk amith added a comment -

        manually verified the patch, with testcase i was not able to assert the new behaviour correctly

        Show
        amithdk amith added a comment - manually verified the patch, with testcase i was not able to assert the new behaviour correctly
        Hide
        amithdk amith added a comment -

        Hi Todd/Nicholas/Uma can u have a look into my patch

        Thanks in Advance

        Show
        amithdk amith added a comment - Hi Todd/Nicholas/Uma can u have a look into my patch Thanks in Advance
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        It seems that setupPipelineForAppendOrRecovery() always return false. So the patch could be simplified to

        +            if (streamerClosed) {
        +              continue;
        +            }
        

        We should also change setupPipelineForAppendOrRecovery() to void. We may do it separately.

        Could you add a unit test?

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - It seems that setupPipelineForAppendOrRecovery() always return false. So the patch could be simplified to + if (streamerClosed) { + continue ; + } We should also change setupPipelineForAppendOrRecovery() to void. We may do it separately. Could you add a unit test?
        Hide
        amithdk amith added a comment -

        Thanks nicholas i will redo the patch and submit a new one

        Show
        amithdk amith added a comment - Thanks nicholas i will redo the patch and submit a new one
        Hide
        amithdk amith added a comment -

        I have added a patch with a test.

        In test I added a assert in the catch block because if a assert is added to check blockStream as null then in failure condition the throwable catches the assertion failed error and testcase passed.

        In catch block I am checking no NPE should be thrown.

        If assertion fails in the catch block then testcase fails due to timeout

        Show
        amithdk amith added a comment - I have added a patch with a test. In test I added a assert in the catch block because if a assert is added to check blockStream as null then in failure condition the throwable catches the assertion failed error and testcase passed. In catch block I am checking no NPE should be thrown. If assertion fails in the catch block then testcase fails due to timeout
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12528206/HDFS-3384_1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 javadoc. The javadoc tool appears to have generated 2 warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestAppendCorruptedBlock

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2486//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2486//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12528206/HDFS-3384_1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestAppendCorruptedBlock +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2486//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2486//console This message is automatically generated.
        Hide
        amithdk amith added a comment -

        Attaching the patch, assert is add in the datastreamer's catch block.

        Show
        amithdk amith added a comment - Attaching the patch, assert is add in the datastreamer's catch block.
        Hide
        amithdk amith added a comment -

        Apache feather symbol is not coming even after granting licence so attaching again

        Show
        amithdk amith added a comment - Apache feather symbol is not coming even after granting licence so attaching again
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12528458/HDFS-3384_2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2494//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2494//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12528458/HDFS-3384_2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2494//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2494//console This message is automatically generated.
        Hide
        umamaheswararao Uma Maheswara Rao G added a comment -

        Hi Amith, Change looks good to me.

        nits:

        DistributedFileSystem fs = cluster.getFileSystem();
        +    Path fileName = new Path("/appendCorruptBlock");
        +    DFSTestUtil.createFile(fs, fileName, 512, (short) 1, 0);
        +    DFSTestUtil.waitReplication(fs, fileName, (short)1);
        +    Assert.assertTrue("File not created", fs.exists(fileName));
        +    /*
        +     * Corrupt the blocks in the cluster
        +     */
        +    corruptBlockInFinalizedDirectory(cluster);

        Here null check for cluster as you have instantiated out side of try. And Better move other code into try after creating cluster. check the other places as well in corruptFinalizedBlock.

        2) "Finalized directory not created yet"
        used in multiple places. create constant and use.

        3) con't we move this test to some existing append test files instead of creating your own test file for single test case?

        Nicholas, do you have any comments?
        I agree that, other than adding assertion in code, I could not find other good way to test this, as Streamer is handling Throwable internally. I am not sure, you will agree with this, please give your opinion.

        Show
        umamaheswararao Uma Maheswara Rao G added a comment - Hi Amith, Change looks good to me. nits: DistributedFileSystem fs = cluster.getFileSystem(); + Path fileName = new Path( "/appendCorruptBlock" ); + DFSTestUtil.createFile(fs, fileName, 512, ( short ) 1, 0); + DFSTestUtil.waitReplication(fs, fileName, ( short )1); + Assert.assertTrue( "File not created" , fs.exists(fileName)); + /* + * Corrupt the blocks in the cluster + */ + corruptBlockInFinalizedDirectory(cluster); Here null check for cluster as you have instantiated out side of try. And Better move other code into try after creating cluster. check the other places as well in corruptFinalizedBlock. 2) "Finalized directory not created yet" used in multiple places. create constant and use. 3) con't we move this test to some existing append test files instead of creating your own test file for single test case? Nicholas, do you have any comments? I agree that, other than adding assertion in code, I could not find other good way to test this, as Streamer is handling Throwable internally. I am not sure, you will agree with this, please give your opinion.
        Hide
        amithdk amith added a comment -

        Attaching a patch

        Show
        amithdk amith added a comment - Attaching a patch
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12530164/HDFS-3384_2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2539//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2539//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530164/HDFS-3384_2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2539//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2539//console This message is automatically generated.
        Hide
        umamaheswararao Uma Maheswara Rao G added a comment -

        +1 on the latest patch!

        Show
        umamaheswararao Uma Maheswara Rao G added a comment - +1 on the latest patch!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12530164/HDFS-3384_2.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / f1a152c
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10533/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12530164/HDFS-3384_2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f1a152c Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10533/console This message was automatically generated.
        Hide
        umamaheswararao Uma Maheswara Rao G added a comment -

        Sorry, for taking longer in pushing this. I missed this JIRA in committing. Since amith is not active now, let me produce new patch based on latest changes.

        Show
        umamaheswararao Uma Maheswara Rao G added a comment - Sorry, for taking longer in pushing this. I missed this JIRA in committing. Since amith is not active now, let me produce new patch based on latest changes.
        Hide
        umamaheswararao Uma Maheswara Rao G added a comment -

        Attached new patch on latest trunk!

        Show
        umamaheswararao Uma Maheswara Rao G added a comment - Attached new patch on latest trunk!
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 27s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 26s There were no new javac warning messages.
        +1 javadoc 9m 30s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 2m 12s The applied patch generated 3 new checkstyle issues (total was 92, now 94).
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 32s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 3m 2s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 native 3m 14s Pre-build of native portion
        -1 hdfs tests 167m 43s Tests failed in hadoop-hdfs.
            210m 12s  



        Reason Tests
        Failed unit tests hadoop.tracing.TestTraceAdmin



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12730731/HDFS-3384-3.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a583a40
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10829/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10829/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10829/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10829/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 27s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 26s There were no new javac warning messages. +1 javadoc 9m 30s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 2m 12s The applied patch generated 3 new checkstyle issues (total was 92, now 94). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 2s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 14s Pre-build of native portion -1 hdfs tests 167m 43s Tests failed in hadoop-hdfs.     210m 12s   Reason Tests Failed unit tests hadoop.tracing.TestTraceAdmin Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12730731/HDFS-3384-3.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a583a40 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10829/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10829/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10829/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10829/console This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Hi Uma Maheswara Rao G, Thanks for taking up old issue.
        You might want to rebase last HDFS-3384_2.patch, one with test and fixed review comments.

        Show
        vinayrpet Vinayakumar B added a comment - Hi Uma Maheswara Rao G , Thanks for taking up old issue. You might want to rebase last HDFS-3384 _2.patch, one with test and fixed review comments.
        Hide
        umamaheswararao Uma Maheswara Rao G added a comment -

        oh Yes. You are right. I have to rebase latest one. Thanks. I will do that in some tome today.

        Show
        umamaheswararao Uma Maheswara Rao G added a comment - oh Yes. You are right. I have to rebase latest one. Thanks. I will do that in some tome today.
        Hide
        umamaheswararao Uma Maheswara Rao G added a comment -

        Vinayakumar B rebased the patch. Also corrected the test case where try-catch should cover the rest of the code path after cluster instantiate.

        Show
        umamaheswararao Uma Maheswara Rao G added a comment - Vinayakumar B rebased the patch. Also corrected the test case where try-catch should cover the rest of the code path after cluster instantiate.
        Hide
        vinayrpet Vinayakumar B added a comment -

        +1,

        Show
        vinayrpet Vinayakumar B added a comment - +1,
        Hide
        vinayrpet Vinayakumar B added a comment -

        Committed to trunk and branch-2.

        Show
        vinayrpet Vinayakumar B added a comment - Committed to trunk and branch-2.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7774 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7774/)
        HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7774 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7774/ ) HDFS-3384 . DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #190 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/190/)
        HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #190 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/190/ ) HDFS-3384 . DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2119 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2119/)
        HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2119 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2119/ ) HDFS-3384 . DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #179 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/179/)
        HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #179 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/179/ ) HDFS-3384 . DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2137 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2137/)
        HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2137 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2137/ ) HDFS-3384 . DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #189 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/189/)
        HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #189 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/189/ ) HDFS-3384 . DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #922 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/922/)
        HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #922 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/922/ ) HDFS-3384 . DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) (vinayakumarb: rev c648317a68891e1c900f04b7a9c98ba40c5faddb) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java

          People

          • Assignee:
            umamaheswararao Uma Maheswara Rao G
            Reporter:
            brahmareddy Brahma Reddy Battula
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development