Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9181

Better handling of exceptions thrown during upgrade shutdown

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Previously in HDFS-7533, a bug was fixed by suppressing exceptions during upgrade shutdown. It may be appropriate as a temporary fix, but it would be better if the exception is handled in some other ways.

      One way to handle it is by emitting a warning message. There could exist other ways to handle it. This jira is created to discuss how to handle this case better.

      Thanks to Daniel Templeton for bringing this up.

      1. HDFS-9181.005.patch
        3 kB
        Wei-Chiu Chuang
      2. HDFS-9181.004.patch
        2 kB
        Wei-Chiu Chuang
      3. HDFS-9181.003.patch
        2 kB
        Wei-Chiu Chuang
      4. HDFS-9181.002.patch
        2 kB
        Wei-Chiu Chuang

        Issue Links

          Activity

          Hide
          templedf Daniel Templeton added a comment -

          My concern is mostly that swallowing a Throwable can have negative consequences. Changing the catch to Exception or rethrowing Errors would be a step in the right direction.

          Show
          templedf Daniel Templeton added a comment - My concern is mostly that swallowing a Throwable can have negative consequences. Changing the catch to Exception or rethrowing Errors would be a step in the right direction.
          Hide
          eepayne Eric Payne added a comment -

          Thanks Wei-Chiu Chuang and Daniel Templeton for bringing this to my attention.

          I am fine with putting out a warning in the case of an exception or throwable during shutdown. I want to emphasize that whatever resolution is decided upon for this JIRA, the functionality of allowing the shutdown thread to finish without hanging should be maintained.

          Wei-Chiu Chuang stated in this comment of HDFS-7533, "It looks to me that catching Throwable and ignore it may cause other issues, for example, ignoring a OOME can be pretty bad."

          Meanwhile, Kihwal Lee pointed out in this comment of HDFS-7533 that "We can check whether a responder is running, but it may be in the process of shutting down. Therefore, a proper check requires additional locking."

          In my opinion, putting in the logic to do additional locking and handling of exceptions is more complicated than is warranted during the datanode shutdown thread. The locking could also cause additional risk and complexity. I do agree that ignoring an OOM may cause a total datanode crash. However, that would be more desirable in this case than a datanode hang, which was happening before HDFS-7533. Will ignoring other throwables also cause the datanode to hang?

          Show
          eepayne Eric Payne added a comment - Thanks Wei-Chiu Chuang and Daniel Templeton for bringing this to my attention. I am fine with putting out a warning in the case of an exception or throwable during shutdown. I want to emphasize that whatever resolution is decided upon for this JIRA, the functionality of allowing the shutdown thread to finish without hanging should be maintained. Wei-Chiu Chuang stated in this comment of HDFS-7533 , "It looks to me that catching Throwable and ignore it may cause other issues, for example, ignoring a OOME can be pretty bad." Meanwhile, Kihwal Lee pointed out in this comment of HDFS-7533 that "We can check whether a responder is running, but it may be in the process of shutting down. Therefore, a proper check requires additional locking." In my opinion, putting in the logic to do additional locking and handling of exceptions is more complicated than is warranted during the datanode shutdown thread. The locking could also cause additional risk and complexity. I do agree that ignoring an OOM may cause a total datanode crash. However, that would be more desirable in this case than a datanode hang, which was happening before HDFS-7533 . Will ignoring other throwables also cause the datanode to hang?
          Hide
          templedf Daniel Templeton added a comment -

          I think you can get what you want by just changing the catch from Throwable to Exception. An uncaught Error will generally bring the whole thing down anyway, so not catching Errors won't impact the hanging behavior. (And if it does, we have bigger issues.)

          Show
          templedf Daniel Templeton added a comment - I think you can get what you want by just changing the catch from Throwable to Exception. An uncaught Error will generally bring the whole thing down anyway, so not catching Errors won't impact the hanging behavior. (And if it does, we have bigger issues.)
          Hide
          eepayne Eric Payne added a comment -

          I think you can get what you want by just changing the catch from Throwable to Exception. An uncaught Error will generally bring the whole thing down anyway, so not catching Errors won't impact the hanging behavior. (And if it does, we have bigger issues.)

          Thanks, Daniel Templeton. I'm fine with catching Exceptions and not Throwables. I would be interested in what Kihwal Lee thinks.

          Show
          eepayne Eric Payne added a comment - I think you can get what you want by just changing the catch from Throwable to Exception. An uncaught Error will generally bring the whole thing down anyway, so not catching Errors won't impact the hanging behavior. (And if it does, we have bigger issues.) Thanks, Daniel Templeton . I'm fine with catching Exceptions and not Throwables. I would be interested in what Kihwal Lee thinks.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks to Daniel Templeton and Eric Payne for the discussion.
          Catching Throwable is generally regarded as an undesirable practice (See http://stackoverflow.com/questions/17333925/is-it-ok-to-catch-throwable-for-performing-cleanup and http://stackoverflow.com/questions/6083248/is-it-a-bad-practice-to-catch-throwable)

          My only question is, if we go for catching the Exception, is it ok to silently ignore it?

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks to Daniel Templeton and Eric Payne for the discussion. Catching Throwable is generally regarded as an undesirable practice (See http://stackoverflow.com/questions/17333925/is-it-ok-to-catch-throwable-for-performing-cleanup and http://stackoverflow.com/questions/6083248/is-it-a-bad-practice-to-catch-throwable ) My only question is, if we go for catching the Exception, is it ok to silently ignore it?
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Guys,

          Since HDFS-7533 tries to catch NPE:

          When datanode is told to shutdown via the dfsadmin command during rolling upgrade, it may not shutdown. This is because not all writers have responder running, but sendOOB() tries anyway. This causes NPE and the shutdown thread dies, halting the shutdown after only shutting down DataXceiverServer.
          

          I'd think that we just change the Throwable to Exception as Daniel suggested, and issue a warning message with the exception's information. For any other thing thrown from the sendOOB step, the behavior will be just like before HDFS-7533.

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Hi Guys, Since HDFS-7533 tries to catch NPE: When datanode is told to shutdown via the dfsadmin command during rolling upgrade, it may not shutdown. This is because not all writers have responder running, but sendOOB() tries anyway. This causes NPE and the shutdown thread dies, halting the shutdown after only shutting down DataXceiverServer. I'd think that we just change the Throwable to Exception as Daniel suggested, and issue a warning message with the exception's information. For any other thing thrown from the sendOOB step, the behavior will be just like before HDFS-7533 . Thanks.
          Hide
          kihwal Kihwal Lee added a comment -

          An uncaught Error will generally bring the whole thing down anyway

          It will bring down the thread, but not the datanode. If that happens, the datanode will probably be in a half-deaf state. In any case, catching Exception should be sufficient.

          Show
          kihwal Kihwal Lee added a comment - An uncaught Error will generally bring the whole thing down anyway It will bring down the thread, but not the datanode. If that happens, the datanode will probably be in a half-deaf state. In any case, catching Exception should be sufficient.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Adding a small patch to catch an Exception instead of a Throwable, and output a message.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Adding a small patch to catch an Exception instead of a Throwable, and output a message.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Update the patch to fix code style issue.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Update the patch to fix code style issue.
          Hide
          templedf Daniel Templeton added a comment -

          Since the logging message doesn't actually provide anyone any useful information, I'd make it TRACE level. No reason to add to the log clutter.

          Show
          templedf Daniel Templeton added a comment - Since the logging message doesn't actually provide anyone any useful information, I'd make it TRACE level. No reason to add to the log clutter.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Use Log.trace() instead of Log.warn().

          Show
          jojochuang Wei-Chiu Chuang added a comment - Use Log.trace() instead of Log.warn().
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 36s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 2s There were no new javac warning messages.
          +1 javadoc 10m 21s There were no new javadoc warning messages.
          -1 release audit 0m 15s The applied patch generated 1 release audit warnings.
          -1 checkstyle 1m 26s The applied patch generated 2 new checkstyle issues (total was 142, now 142).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 18s Pre-build of native portion
          -1 hdfs tests 235m 59s Tests failed in hadoop-hdfs.
              282m 38s  



          Reason Tests
          Failed unit tests hadoop.hdfs.web.TestWebHdfsContentLength
            hadoop.hdfs.server.namenode.TestCheckpoint
            hadoop.hdfs.web.TestWebHdfsTimeouts
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.security.TestDelegationTokenForProxyUser



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765273/HDFS-9181.002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 6d5713a
          Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12820/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12820/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12820/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12820/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12820/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 2s There were no new javac warning messages. +1 javadoc 10m 21s There were no new javadoc warning messages. -1 release audit 0m 15s The applied patch generated 1 release audit warnings. -1 checkstyle 1m 26s The applied patch generated 2 new checkstyle issues (total was 142, now 142). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 18s Pre-build of native portion -1 hdfs tests 235m 59s Tests failed in hadoop-hdfs.     282m 38s   Reason Tests Failed unit tests hadoop.hdfs.web.TestWebHdfsContentLength   hadoop.hdfs.server.namenode.TestCheckpoint   hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.security.TestDelegationTokenForProxyUser Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765273/HDFS-9181.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 6d5713a Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12820/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12820/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12820/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12820/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12820/console This message was automatically generated.
          Hide
          yzhangal Yongjun Zhang added a comment -

          HI Wei-Chiu Chuang,

          Would you please remove // Issue an warning., +1 after that.

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - HI Wei-Chiu Chuang , Would you please remove // Issue an warning. , +1 after that. Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 48s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 54s There were no new javac warning messages.
          +1 javadoc 10m 33s There were no new javadoc warning messages.
          -1 release audit 0m 15s The applied patch generated 1 release audit warnings.
          -1 checkstyle 1m 27s The applied patch generated 2 new checkstyle issues (total was 142, now 142).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 30s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 14s Pre-build of native portion
          -1 hdfs tests 201m 1s Tests failed in hadoop-hdfs.
              247m 53s  



          Reason Tests
          Failed unit tests hadoop.hdfs.web.TestWebHdfsContentLength
            hadoop.hdfs.server.blockmanagement.TestBlockManager
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.web.TestWebHdfsTimeouts



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765309/HDFS-9181.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 1bca1bb
          Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12824/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12824/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12824/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12824/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12824/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 48s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 54s There were no new javac warning messages. +1 javadoc 10m 33s There were no new javadoc warning messages. -1 release audit 0m 15s The applied patch generated 1 release audit warnings. -1 checkstyle 1m 27s The applied patch generated 2 new checkstyle issues (total was 142, now 142). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 14s Pre-build of native portion -1 hdfs tests 201m 1s Tests failed in hadoop-hdfs.     247m 53s   Reason Tests Failed unit tests hadoop.hdfs.web.TestWebHdfsContentLength   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.web.TestWebHdfsTimeouts Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765309/HDFS-9181.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1bca1bb Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12824/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12824/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12824/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12824/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12824/console This message was automatically generated.
          Hide
          eepayne Eric Payne added a comment -

          Wei-Chiu Chuang, Thanks for the patch.
          +1 (non-binding)
          LGTM

          Show
          eepayne Eric Payne added a comment - Wei-Chiu Chuang , Thanks for the patch. +1 (non-binding) LGTM
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          removed a line of comment per Yongjun's comment.

          Show
          jojochuang Wei-Chiu Chuang added a comment - removed a line of comment per Yongjun's comment.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 patch 0m 0s The patch command could not apply the patch during dryrun.



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765601/HDFS-9123.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 1107bd3
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12864/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765601/HDFS-9123.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1107bd3 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12864/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Submitted the wrong patch. This one is correct.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Submitted the wrong patch. This one is correct.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 20m 1s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 9m 8s There were no new javac warning messages.
          +1 javadoc 11m 42s There were no new javadoc warning messages.
          -1 release audit 0m 20s The applied patch generated 1 release audit warnings.
          -1 checkstyle 1m 42s The applied patch generated 2 new checkstyle issues (total was 142, now 142).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 39s mvn install still works.
          +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse.
          +1 findbugs 2m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 37s Pre-build of native portion
          -1 hdfs tests 197m 56s Tests failed in hadoop-hdfs.
              249m 36s  



          Reason Tests
          Failed unit tests hadoop.fs.TestWebHdfsFileContextMainOperations
            hadoop.hdfs.server.namenode.ha.TestRequestHedgingProxyProvider
            hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765606/HDFS-9181.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 1107bd3
          Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12865/artifact/patchprocess/patchReleaseAuditProblems.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12865/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12865/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12865/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12865/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 20m 1s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 9m 8s There were no new javac warning messages. +1 javadoc 11m 42s There were no new javadoc warning messages. -1 release audit 0m 20s The applied patch generated 1 release audit warnings. -1 checkstyle 1m 42s The applied patch generated 2 new checkstyle issues (total was 142, now 142). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse. +1 findbugs 2m 51s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 37s Pre-build of native portion -1 hdfs tests 197m 56s Tests failed in hadoop-hdfs.     249m 36s   Reason Tests Failed unit tests hadoop.fs.TestWebHdfsFileContextMainOperations   hadoop.hdfs.server.namenode.ha.TestRequestHedgingProxyProvider   hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765606/HDFS-9181.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 1107bd3 Release Audit https://builds.apache.org/job/PreCommit-HDFS-Build/12865/artifact/patchprocess/patchReleaseAuditProblems.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12865/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12865/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12865/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12865/console This message was automatically generated.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Refactor shutdown() to reduce its length.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Refactor shutdown() to reduce its length.
          Hide
          yzhangal Yongjun Zhang added a comment -

          I committed rev 004 to trunk and branch-2. Thanks Wei-Chiu for the contribution, and all for the review.

          Just saw Wei-Chiu uoloaded a new rev 005 with some refactoring. Suggest to create follow-up jira and do the refactoring when fixing another issue in same area.

          Show
          yzhangal Yongjun Zhang added a comment - I committed rev 004 to trunk and branch-2. Thanks Wei-Chiu for the contribution, and all for the review. Just saw Wei-Chiu uoloaded a new rev 005 with some refactoring. Suggest to create follow-up jira and do the refactoring when fixing another issue in same area.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8601 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8601/)
          HDFS-9181. Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8601 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8601/ ) HDFS-9181 . Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #515 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/515/)
          HDFS-9181. Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #515 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/515/ ) HDFS-9181 . Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Hide
          eepayne Eric Payne added a comment -

          I committed rev 004 to trunk and branch-2. Thanks Wei-Chiu for the contribution, and all for the review.

          Just saw Wei-Chiu uploaded a new rev 005 with some refactoring. Suggest to create follow-up jira and do the refactoring when fixing another issue in same area.

          I agree with Yongjun Zhang's suggestion to make the refactoring task a separate JIRA. I believe that it is much easier to review changes when they are focused on one issue. In fact, if refactoring is to be done, I would rather that it be done by itself rather than with any other changes.

          Show
          eepayne Eric Payne added a comment - I committed rev 004 to trunk and branch-2. Thanks Wei-Chiu for the contribution, and all for the review. Just saw Wei-Chiu uploaded a new rev 005 with some refactoring. Suggest to create follow-up jira and do the refactoring when fixing another issue in same area. I agree with Yongjun Zhang 's suggestion to make the refactoring task a separate JIRA. I believe that it is much easier to review changes when they are focused on one issue. In fact, if refactoring is to be done, I would rather that it be done by itself rather than with any other changes.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2449 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2449/)
          HDFS-9181. Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2449 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2449/ ) HDFS-9181 . Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1242 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1242/)
          HDFS-9181. Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1242 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1242/ ) HDFS-9181 . Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/477/)
          HDFS-9181. Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #477 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/477/ ) HDFS-9181 . Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2415 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2415/)
          HDFS-9181. Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2415 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2415/ ) HDFS-9181 . Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #505 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/505/)
          HDFS-9181. Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #505 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/505/ ) HDFS-9181 . Better handling of exceptions thrown during upgrade shutdown. (yzhang: rev c11fc8a1be222f870cded0b24736387e44cc788c) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeExit.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java

            People

            • Assignee:
              jojochuang Wei-Chiu Chuang
              Reporter:
              jojochuang Wei-Chiu Chuang
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development