HBase
  1. HBase
  2. HBASE-5963

ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.94.0, 0.95.2
    • Fix Version/s: 0.94.1, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Bigtop 0.4.0-incubating-SNAPSHOT, Hadoop 2.0.0-alpha-SNAPSHOT, HBase 0.94.0-SNAPSHOT

    • Hadoop Flags:
      Reviewed

      Description

      12/05/08 19:49:26 INFO regionserver.HRegionServer: STOPPED: Failed initialization
      Exception in thread "main" java.lang.ClassCastException: org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer cannot be cast to java.lang.Thread
      	at org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:181)
      	at org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:82)
      	at org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3601)
      	at org.apache.hadoop.hbase.regionserver.HRegionServer.startRegionServer(HRegionServer.java:3585)
      	at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:61)
      	at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:75)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      	at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
      	at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3645)
      
      1. 5963-v2.txt
        6 kB
        Ted Yu
      2. 5963-v3.txt
        6 kB
        Ted Yu
      3. HBASE-5963.patch
        5 kB
        Andrew Purtell
      4. HBASE-5963.patch
        3 kB
        Andrew Purtell

        Activity

        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #27 (See https://builds.apache.org/job/HBase-0.94-security/27/)
        HBASE-5963 ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread (Revision 1336336)

        Result = SUCCESS
        tedyu :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ShutdownHookManager.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #27 (See https://builds.apache.org/job/HBase-0.94-security/27/ ) HBASE-5963 ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread (Revision 1336336) Result = SUCCESS tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ShutdownHookManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #197 (See https://builds.apache.org/job/HBase-TRUNK-security/197/)
        HBASE-5963 ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread (Revision 1336334)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/ShutdownHookManager.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #197 (See https://builds.apache.org/job/HBase-TRUNK-security/197/ ) HBASE-5963 ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread (Revision 1336334) Result = FAILURE tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/ShutdownHookManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #185 (See https://builds.apache.org/job/HBase-0.94/185/)
        HBASE-5963 ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread (Revision 1336336)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ShutdownHookManager.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #185 (See https://builds.apache.org/job/HBase-0.94/185/ ) HBASE-5963 ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread (Revision 1336336) Result = FAILURE tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/ShutdownHookManager.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2859 (See https://builds.apache.org/job/HBase-TRUNK/2859/)
        HBASE-5963 ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread (Revision 1336334)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/ShutdownHookManager.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2859 (See https://builds.apache.org/job/HBase-TRUNK/2859/ ) HBASE-5963 ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread (Revision 1336334) Result = FAILURE tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/ShutdownHook.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/ShutdownHookManager.java
        Hide
        Ted Yu added a comment -

        Integrated to 0.94 and trunk.

        Thanks for providing the initial patch and review, Andy.

        Show
        Ted Yu added a comment - Integrated to 0.94 and trunk. Thanks for providing the initial patch and review, Andy.
        Hide
        Andrew Purtell added a comment -

        +1 for commit, nice work Ted.

        Show
        Andrew Purtell added a comment - +1 for commit, nice work Ted.
        Hide
        Andrew Purtell added a comment -

        I've been testing with 0.94 branch not TRUNK.

        Retesting with a fresh checkout plus only HBASE-5964 applied looks good Ted.

        Show
        Andrew Purtell added a comment - I've been testing with 0.94 branch not TRUNK. Retesting with a fresh checkout plus only HBASE-5964 applied looks good Ted.
        Hide
        Ted Yu added a comment -

        I wasn't able to reproduce the test failure:

        mvn test -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT -PlocalTests -Dtest=TestRegionRebalancing
        ...
        Running org.apache.hadoop.hbase.TestRegionRebalancing
        Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.049 sec
        ...
        [INFO] Total time: 54.616s
        

        Also, the 'Failed suppression of fs shutdown hook' message should be at line 205 of ShutdownHook.java

        @Andy:
        Please give me a little more clue.

        Show
        Ted Yu added a comment - I wasn't able to reproduce the test failure: mvn test -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT -PlocalTests -Dtest=TestRegionRebalancing ... Running org.apache.hadoop.hbase.TestRegionRebalancing Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.049 sec ... [INFO] Total time: 54.616s Also, the 'Failed suppression of fs shutdown hook' message should be at line 205 of ShutdownHook.java @Andy: Please give me a little more clue.
        Hide
        Andrew Purtell added a comment -

        These test pass with the "v1" patch. I think the difference is in Hadoop 1.x there is a cacheFinalizer per filesystem impl, but in 2.x there is one for all filesystems (for the user).

        Show
        Andrew Purtell added a comment - These test pass with the "v1" patch. I think the difference is in Hadoop 1.x there is a cacheFinalizer per filesystem impl, but in 2.x there is one for all filesystems (for the user).
        Hide
        Andrew Purtell added a comment -

        Patch v3 fails TestRegionRebalancing (and others):

        [...]
        Caused by: java.lang.RuntimeException: Failed suppression of fs shutdown hook: Thread[Thread-74,5,main]
        	at org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:197)
        	at org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:83)
        	at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:191)
        	at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:422)
        	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:196)
        	... 28 more
        
        Show
        Andrew Purtell added a comment - Patch v3 fails TestRegionRebalancing (and others): [...] Caused by: java.lang.RuntimeException: Failed suppression of fs shutdown hook: Thread[Thread-74,5,main] at org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:197) at org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:83) at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:191) at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:422) at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:196) ... 28 more
        Hide
        Andrew Purtell added a comment -

        I am also testing the v3 patch now with HBASE-5964 and HBASE-5966 also applied.

        Show
        Andrew Purtell added a comment - I am also testing the v3 patch now with HBASE-5964 and HBASE-5966 also applied.
        Hide
        Ted Yu added a comment -

        With the help from HBASE-5964 and HBASE-5966, I saw TestRowProcessorEndpoint pass against hadoop 2.0

        Show
        Ted Yu added a comment - With the help from HBASE-5964 and HBASE-5966 , I saw TestRowProcessorEndpoint pass against hadoop 2.0
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526111/5963-v3.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1810//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1810//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1810//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526111/5963-v3.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1810//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1810//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1810//console This message is automatically generated.
        Hide
        Ted Yu added a comment -

        Patch v3 incorporates Andy's feedback about using ShutdownHookManager.addShutdownHook

        Show
        Ted Yu added a comment - Patch v3 incorporates Andy's feedback about using ShutdownHookManager.addShutdownHook
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526103/5963-v2.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1808//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1808//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1808//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526103/5963-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1808//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1808//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1808//console This message is automatically generated.
        Hide
        Ted Yu added a comment -

        @Andy:
        I am running the following on Linux:

        nohup mvn test -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT -PrunAllTests > suite.txt &
        

        Will check results tomorrow.
        For hadoop 1.0, PreCommit build 1808 would tell us the results.

        For ShutdownHookManagerV2.addShutdownHook(), current code reflects what was in patch v1. Your suggestion is plausible.

        Show
        Ted Yu added a comment - @Andy: I am running the following on Linux: nohup mvn test -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT -PrunAllTests > suite.txt & Will check results tomorrow. For hadoop 1.0, PreCommit build 1808 would tell us the results. For ShutdownHookManagerV2.addShutdownHook(), current code reflects what was in patch v1. Your suggestion is plausible.
        Hide
        Andrew Purtell added a comment -

        Additionally, shouldn't addShutdownHook in ShutdownHookManagerV2 delegate to the Hadoop 2.x shutdown hook manager?

        Show
        Andrew Purtell added a comment - Additionally, shouldn't addShutdownHook in ShutdownHookManagerV2 delegate to the Hadoop 2.x shutdown hook manager?
        Hide
        Andrew Purtell added a comment -

        Patch v2 is plausible.

        Did you run unit tests with -Dhadoop.profile=23 and -Dhadoop.version=2.0.0-snapshot ? They pass except for the MR ones? And tests pass for Hadoop 1?

        Show
        Andrew Purtell added a comment - Patch v2 is plausible. Did you run unit tests with -Dhadoop.profile=23 and -Dhadoop.version=2.0.0-snapshot ? They pass except for the MR ones? And tests pass for Hadoop 1?
        Hide
        Ted Yu added a comment -

        Patch v2 introduces ShutdownHookManager shim.

        Please review.

        Show
        Ted Yu added a comment - Patch v2 introduces ShutdownHookManager shim. Please review.
        Hide
        Andrew Purtell added a comment -

        The bulk of the work here isn't managing the shutdown hook, it is dealing with the different Hadoop versions through special casing and reflection. If you'd like to make a more involved change here that introduces a shim and works more nicely with 2.0, that would be great. I opted for lower risk of breaking 1.0, CDH, etc. It's definitely a debatable choice.

        Show
        Andrew Purtell added a comment - The bulk of the work here isn't managing the shutdown hook, it is dealing with the different Hadoop versions through special casing and reflection. If you'd like to make a more involved change here that introduces a shim and works more nicely with 2.0, that would be great. I opted for lower risk of breaking 1.0, CDH, etc. It's definitely a debatable choice.
        Hide
        Ted Yu added a comment -

        ShutdownHookManager from hadoop 2.0 is only 181 lines long.
        It shouldn't be hard to produce our ShutdownHookManager shim.

        Show
        Ted Yu added a comment - ShutdownHookManager from hadoop 2.0 is only 181 lines long. It shouldn't be hard to produce our ShutdownHookManager shim.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526076/HBASE-5963.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1807//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1807//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1807//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526076/HBASE-5963.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1807//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1807//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1807//console This message is automatically generated.
        Hide
        Andrew Purtell added a comment -

        Ignore Hadoop QA. To verify the patch, you'll have to run tests locally with

        mvn -PlocalTests -Psecurity \
          -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \
          clean test
        

        Note that all of the MR based tests are currently broken, I opened HBASE-5966.

        Show
        Andrew Purtell added a comment - Ignore Hadoop QA. To verify the patch, you'll have to run tests locally with mvn -PlocalTests -Psecurity \ -Dhadoop.profile=23 -Dhadoop.version=2.0.0-SNAPSHOT \ clean test Note that all of the MR based tests are currently broken, I opened HBASE-5966 .
        Hide
        Andrew Purtell added a comment - - edited

        Latest patch seems to do the right thing according to unit tests without breaking older functionality.

        However, what we are doing here is yanking the HDFS client finalizer Runnable away from the Hadoop 2.x ShutdownHookManager and adding it – wrapped with our existing strategy here – to the Runtime shutdown hook list, to perhaps run concurrently with the rest of the ShutdownHookManager registered Runnables. I don't think this more problematic than the rest of this horrible code. The solution is, someday, to leave Hadoop 1.x behind and transition to using the 2.x ShutdownHookManager.

        Show
        Andrew Purtell added a comment - - edited Latest patch seems to do the right thing according to unit tests without breaking older functionality. However, what we are doing here is yanking the HDFS client finalizer Runnable away from the Hadoop 2.x ShutdownHookManager and adding it – wrapped with our existing strategy here – to the Runtime shutdown hook list, to perhaps run concurrently with the rest of the ShutdownHookManager registered Runnables. I don't think this more problematic than the rest of this horrible code. The solution is, someday, to leave Hadoop 1.x behind and transition to using the 2.x ShutdownHookManager.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526060/HBASE-5963.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop23. The patch compiles against the hadoop 0.23.x profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1804//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1804//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1804//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526060/HBASE-5963.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop23. The patch compiles against the hadoop 0.23.x profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1804//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1804//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1804//console This message is automatically generated.
        Hide
        Andrew Purtell added a comment - - edited

        The attached patch gets past the problem but I'm not able to test it yet due to another issue I'm about to file another JIRA for.

        Edit: I mean, I can see that it gets past the ClassCastException, and the ShutdownHookManager removes the client finalizer successfully, but I have not yet been able to observe if normal RS shutdown is ok.

        Show
        Andrew Purtell added a comment - - edited The attached patch gets past the problem but I'm not able to test it yet due to another issue I'm about to file another JIRA for. Edit: I mean, I can see that it gets past the ClassCastException, and the ShutdownHookManager removes the client finalizer successfully, but I have not yet been able to observe if normal RS shutdown is ok.
        Hide
        Andrew Purtell added a comment -

        Hadoop 2.0 changes the clientFinalizer from a Thread to a Runnable. We can catch the ClassCastException and create a Thread with that Runnable, and then call ShutdownHookManager by reflection to unregister the clientFinalizer.

        Show
        Andrew Purtell added a comment - Hadoop 2.0 changes the clientFinalizer from a Thread to a Runnable. We can catch the ClassCastException and create a Thread with that Runnable, and then call ShutdownHookManager by reflection to unregister the clientFinalizer.
        Hide
        Andrew Purtell added a comment -

        No, it's not that simple because our usage here is more like "ShutdownHook extends FileSystem.Cache".

        Show
        Andrew Purtell added a comment - No, it's not that simple because our usage here is more like "ShutdownHook extends FileSystem.Cache".
        Hide
        Andrew Purtell added a comment -

        We could check for the existence of ShutdownHookManager and just use it, falling back to the old method otherwise.

        Show
        Andrew Purtell added a comment - We could check for the existence of ShutdownHookManager and just use it, falling back to the old method otherwise.
        Hide
        Andrew Purtell added a comment -

        How about catching exceptions on ShutdownHook.install and retrying with ShutdownHookManager, using reflection? This the ShutdownHook class becomes the shim.

        Show
        Andrew Purtell added a comment - How about catching exceptions on ShutdownHook.install and retrying with ShutdownHookManager, using reflection? This the ShutdownHook class becomes the shim.
        Hide
        Ted Yu added a comment -

        hadoop 2.0 has its own ShutdownHookManager
        We need to provide shim which hides Java Runtime w.r.t. shutdown hook management.

        Show
        Ted Yu added a comment - hadoop 2.0 has its own ShutdownHookManager We need to provide shim which hides Java Runtime w.r.t. shutdown hook management.

          People

          • Assignee:
            Ted Yu
            Reporter:
            Andrew Purtell
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development