HBase
  1. HBase
  2. HBASE-10338

Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Runtime exception is being thrown when AccessController CP is used with region server. This is happening as region server co processor host is created before zookeeper is initialized in region server.

      1. 10338.1.patch
        5 kB
        Andrew Purtell
      2. 10338.1.patch
        6 kB
        Andrew Purtell
      3. 10338.1-0.96.patch
        5 kB
        Andrew Purtell
      4. 10338.1-0.98.patch
        5 kB
        Andrew Purtell
      5. HBASE-10338_addendum.patch
        0.7 kB
        Vandana Ayyalasomayajula
      6. HBASE-10338.0.patch
        3 kB
        Vandana Ayyalasomayajula

        Activity

        Hide
        Andrew Purtell added a comment -

        Good catch, we've missed thus far hooking the AC into the RS level with RegionServerCoprocessorHost.

        Show
        Andrew Purtell added a comment - Good catch, we've missed thus far hooking the AC into the RS level with RegionServerCoprocessorHost.
        Hide
        Andrew Purtell added a comment -

        Let me check this locally and then will commit after.

        Show
        Andrew Purtell added a comment - Let me check this locally and then will commit after.
        Hide
        Andrew Purtell added a comment - - edited

        If you don't mind Vandana Ayyalasomayajula, I would also like to add a unit test to catch this on top of your patch.
        Edit: Maybe changes to TestAccessController will be enough, it didn't catch it but should have.

        Show
        Andrew Purtell added a comment - - edited If you don't mind Vandana Ayyalasomayajula , I would also like to add a unit test to catch this on top of your patch. Edit: Maybe changes to TestAccessController will be enough, it didn't catch it but should have.
        Hide
        stack added a comment -

        +1 on patch. Test would be good too. (+1 for 0.96 if we have this issue there too)

        Show
        stack added a comment - +1 on patch. Test would be good too. (+1 for 0.96 if we have this issue there too)
        Hide
        Vandana Ayyalasomayajula added a comment -

        Andrew Purtell This happens only when the coprocessor uses zookeeper in its start method. I made changes to security util class, so that this particular case is caught. Do you think we need separate unit test for this case ?

        Show
        Vandana Ayyalasomayajula added a comment - Andrew Purtell This happens only when the coprocessor uses zookeeper in its start method. I made changes to security util class, so that this particular case is caught. Do you think we need separate unit test for this case ?
        Hide
        Vandana Ayyalasomayajula added a comment -

        This issue is seen on 0.96 too.

        Show
        Vandana Ayyalasomayajula added a comment - This issue is seen on 0.96 too.
        Hide
        Andrew Purtell added a comment -

        I'm thinking just this to address how this slipped through. TestAccessController should insure that the AC is added to all system coprocessor lists, which are set up elsewhere.

        diff --git hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/Te
        index 9b8dd6e..ef56384 100644
        --- hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAcces
        +++ hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAcces
        @@ -59,6 +59,7 @@ import org.apache.hadoop.hbase.client.Put;
         import org.apache.hadoop.hbase.client.Result;
         import org.apache.hadoop.hbase.client.ResultScanner;
         import org.apache.hadoop.hbase.client.Scan;
        +import org.apache.hadoop.hbase.coprocessor.CoprocessorHost;
         import org.apache.hadoop.hbase.coprocessor.CoprocessorService;
         import org.apache.hadoop.hbase.coprocessor.MasterCoprocessorEnvironment;
         import org.apache.hadoop.hbase.coprocessor.ObserverContext;
        @@ -157,6 +158,17 @@ public class TestAccessController extends SecureTestUtil {
           private static RegionServerCoprocessorEnvironment RSCP_ENV;
           private RegionCoprocessorEnvironment RCP_ENV;
         
        +  static void verifyConfiguration(Configuration conf) {
        +    if (!conf.get(CoprocessorHost.MASTER_COPROCESSOR_CONF_KEY)
        +            .contains(AccessController.class.getName())
        +        && conf.get(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY)
        +            .contains(AccessController.class.getName())
        +        && conf.get(CoprocessorHost.REGIONSERVER_COPROCESSOR_CONF_KEY)
        +            .contains(AccessController.class.getName())) {
        +      throw new RuntimeException("AccessController is missing from a system cop
        +    }
        +  }
        +
           @BeforeClass
           public static void setupBeforeClass() throws Exception {
             // setup configuration
        @@ -168,6 +180,9 @@ public class TestAccessController extends SecureTestUtil {
               "org.apache.hadoop.hbase.master.snapshot.SnapshotLogCleaner");
             // Enable security
             SecureTestUtil.enableSecurity(conf);
        +    // Verify enableSecurity sets up what we require
        +    verifyConfiguration(conf);
        +
             // Enable EXEC permission checking
             conf.setBoolean(AccessController.EXEC_PERMISSION_CHECKS_KEY, true);
        
        Show
        Andrew Purtell added a comment - I'm thinking just this to address how this slipped through. TestAccessController should insure that the AC is added to all system coprocessor lists, which are set up elsewhere. diff --git hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/Te index 9b8dd6e..ef56384 100644 --- hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAcces +++ hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAcces @@ -59,6 +59,7 @@ import org.apache.hadoop.hbase.client.Put; import org.apache.hadoop.hbase.client.Result; import org.apache.hadoop.hbase.client.ResultScanner; import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.coprocessor.CoprocessorHost; import org.apache.hadoop.hbase.coprocessor.CoprocessorService; import org.apache.hadoop.hbase.coprocessor.MasterCoprocessorEnvironment; import org.apache.hadoop.hbase.coprocessor.ObserverContext; @@ -157,6 +158,17 @@ public class TestAccessController extends SecureTestUtil { private static RegionServerCoprocessorEnvironment RSCP_ENV; private RegionCoprocessorEnvironment RCP_ENV; + static void verifyConfiguration(Configuration conf) { + if (!conf.get(CoprocessorHost.MASTER_COPROCESSOR_CONF_KEY) + .contains(AccessController.class.getName()) + && conf.get(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY) + .contains(AccessController.class.getName()) + && conf.get(CoprocessorHost.REGIONSERVER_COPROCESSOR_CONF_KEY) + .contains(AccessController.class.getName())) { + throw new RuntimeException("AccessController is missing from a system cop + } + } + @BeforeClass public static void setupBeforeClass() throws Exception { // setup configuration @@ -168,6 +180,9 @@ public class TestAccessController extends SecureTestUtil { "org.apache.hadoop.hbase.master.snapshot.SnapshotLogCleaner"); // Enable security SecureTestUtil.enableSecurity(conf); + // Verify enableSecurity sets up what we require + verifyConfiguration(conf); + // Enable EXEC permission checking conf.setBoolean(AccessController.EXEC_PERMISSION_CHECKS_KEY, true);
        Hide
        Andrew Purtell added a comment -

        ... with one more set of parens to enclose the negation for style reasons but you get the idea.

        Show
        Andrew Purtell added a comment - ... with one more set of parens to enclose the negation for style reasons but you get the idea.
        Hide
        stack added a comment -

        Good by me.

        Show
        stack added a comment - Good by me.
        Hide
        Vandana Ayyalasomayajula added a comment -

        I think if we need a verify method, then it should be in the SecureTestUtil class, so that it can be used by other security related test classes.

        Show
        Vandana Ayyalasomayajula added a comment - I think if we need a verify method, then it should be in the SecureTestUtil class, so that it can be used by other security related test classes.
        Hide
        stack added a comment -

        /me I like Vandana Ayyalasomayajula's suggestion

        Show
        stack added a comment - /me I like Vandana Ayyalasomayajula 's suggestion
        Hide
        Andrew Purtell added a comment -

        Also, TestAccessController has coverage for the RegionServerObserver interface methods, but does so by calling them directly, not via the RegionServer, so it didn't catch the AC was not installed into the RegionServerCoprocessorHost previously. Not needed now but could be a reasonable follow up.

        Show
        Andrew Purtell added a comment - Also, TestAccessController has coverage for the RegionServerObserver interface methods, but does so by calling them directly, not via the RegionServer, so it didn't catch the AC was not installed into the RegionServerCoprocessorHost previously. Not needed now but could be a reasonable follow up.
        Hide
        Andrew Purtell added a comment -

        I think if we need a verify method, then it should be in the SecureTestUtil class, so that it can be used by other security related test classes

        I don't feel that is appropriate. TestAccessController specifically needs to test that the AC is installed. Also, putting this check into SecureTestUtil is redundant, it would serve no purpose but for documentation, could just put in a comment then.

        Show
        Andrew Purtell added a comment - I think if we need a verify method, then it should be in the SecureTestUtil class, so that it can be used by other security related test classes I don't feel that is appropriate. TestAccessController specifically needs to test that the AC is installed. Also, putting this check into SecureTestUtil is redundant, it would serve no purpose but for documentation, could just put in a comment then.
        Hide
        Andrew Purtell added a comment -

        For your consideration

        Show
        Andrew Purtell added a comment - For your consideration
        Hide
        Vandana Ayyalasomayajula added a comment -

        Seems like the other test classes do not quite require the AccessController class in the region server. So we can have the method which is verifying the AccessController being present in the master, region server and region to be in the TestAccessController class itself.

        Show
        Vandana Ayyalasomayajula added a comment - Seems like the other test classes do not quite require the AccessController class in the region server. So we can have the method which is verifying the AccessController being present in the master, region server and region to be in the TestAccessController class itself.
        Hide
        Andrew Purtell added a comment -

        Ok I will commit the .1 patch with credit to Vandana shortly to trunk, 0.98, and 0.96 unless objection.

        Show
        Andrew Purtell added a comment - Ok I will commit the .1 patch with credit to Vandana shortly to trunk, 0.98, and 0.96 unless objection.
        Hide
        Vandana Ayyalasomayajula added a comment -
        Show
        Vandana Ayyalasomayajula added a comment - Thanks Andrew Purtell .
        Hide
        Andrew Purtell added a comment -

        I needed to backport the trunk changes to both 0.96 and 0.98 in slightly different ways. Attaching the result. TestAccessController passes locally on all branches. I will run the full test suite locally before committing.

        Show
        Andrew Purtell added a comment - I needed to backport the trunk changes to both 0.96 and 0.98 in slightly different ways. Attaching the result. TestAccessController passes locally on all branches. I will run the full test suite locally before committing.
        Hide
        Andrew Purtell added a comment -

        Committed to trunk, 0.98, and 0.96 branches. Huge thanks for reporting this and providing a patch Vandana Ayyalasomayajula

        Show
        Andrew Purtell added a comment - Committed to trunk, 0.98, and 0.96 branches. Huge thanks for reporting this and providing a patch Vandana Ayyalasomayajula
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.98 #81 (See https://builds.apache.org/job/HBase-0.98/81/)
        HBASE-10338. Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558261)

        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.98 #81 (See https://builds.apache.org/job/HBase-0.98/81/ ) HBASE-10338 . Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558261) /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.96 #258 (See https://builds.apache.org/job/hbase-0.96/258/)
        HBASE-10338. Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558262)

        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.96 #258 (See https://builds.apache.org/job/hbase-0.96/258/ ) HBASE-10338 . Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558262) /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #4820 (See https://builds.apache.org/job/HBase-TRUNK/4820/)
        HBASE-10338. Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558260)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4820 (See https://builds.apache.org/job/HBase-TRUNK/4820/ ) HBASE-10338 . Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558260) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #75 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/75/)
        HBASE-10338. Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558261)

        • /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #75 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/75/ ) HBASE-10338 . Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558261) /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.96-hadoop2 #176 (See https://builds.apache.org/job/hbase-0.96-hadoop2/176/)
        HBASE-10338. Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558262)

        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.96-hadoop2 #176 (See https://builds.apache.org/job/hbase-0.96-hadoop2/176/ ) HBASE-10338 . Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558262) /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #54 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/54/)
        HBASE-10338. Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558260)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #54 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/54/ ) HBASE-10338 . Region server fails to start with AccessController coprocessor if installed into RegionServerCoprocessorHost (Vandana Ayyalasomayajula) (apurtell: rev 1558260) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java
        Hide
        stack added a comment -

        This patch destabilized the 0.96 builds. Any chance of taking a look Vandana. See here: https://builds.apache.org/job/hbase-0.96/ See how since this patch went in org.apache.hadoop.hbase.regionserver.TestRSKilledWhenInitializing.testRSTermnationAfterRegisteringToMasterBeforeCreatingEphemeralNod has started failing.

        Thanks.

        Show
        stack added a comment - This patch destabilized the 0.96 builds. Any chance of taking a look Vandana. See here: https://builds.apache.org/job/hbase-0.96/ See how since this patch went in org.apache.hadoop.hbase.regionserver.TestRSKilledWhenInitializing.testRSTermnationAfterRegisteringToMasterBeforeCreatingEphemeralNod has started failing. Thanks.
        Hide
        Andrew Purtell added a comment -

        I didn't see that test failure when running the 0.96 suite before commit, so it's likely a timing issue / race.

        Show
        Andrew Purtell added a comment - I didn't see that test failure when running the 0.96 suite before commit, so it's likely a timing issue / race.
        Hide
        Andrew Purtell added a comment -

        No, I stand corrected.

        Here is the core change to 0.96:

        Index: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        ===================================================================
        --- hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java	(revision 1558241)
        +++ hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java	(working copy)
        @@ -574,7 +574,6 @@
                 abort("Uncaught exception in service thread " + t.getName(), e);
               }
             };
        -    this.rsHost = new RegionServerCoprocessorHost(this, this.conf);
         
             this.distributedLogReplay = this.conf.getBoolean(HConstants.DISTRIBUTED_LOG_REPLAY_KEY,
               HConstants.DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG);
        @@ -791,6 +790,11 @@
                 }
               }
         
        +      // Initialize the RegionServerCoprocessorHost now that our ephemeral
        +      // node was created by reportForDuty, in case any coprocessors want
        +      // to use ZooKeeper
        +      this.rsHost = new RegionServerCoprocessorHost(this, this.conf);
        +
               if (!this.stopped && isHealthy()){
                 // start the snapshot handler, since the server is ready to run
                 this.snapshotManager.start();
        

        Looks like moving the initialization has left something uninitialized during the test because when the abort happens there is what looks like an unexpected NPE:

        2014-01-21 23:16:22,279 FATAL [RS:0;vesta:49655] regionserver.HRegionServer(1733): ABORTING region server vesta.apache.org,49655,1390346181063: Unhandled: null
        java.lang.NullPointerException
        	at org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1665)
        	at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1761)
        	at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.abortRegionServer(MiniHBaseCluster.java:173)
        	at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$100(MiniHBaseCluster.java:107)
        	at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$2.run(MiniHBaseCluster.java:166)
        
        Show
        Andrew Purtell added a comment - No, I stand corrected. Here is the core change to 0.96: Index: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java =================================================================== --- hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (revision 1558241) +++ hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (working copy) @@ -574,7 +574,6 @@ abort("Uncaught exception in service thread " + t.getName(), e); } }; - this.rsHost = new RegionServerCoprocessorHost(this, this.conf); this.distributedLogReplay = this.conf.getBoolean(HConstants.DISTRIBUTED_LOG_REPLAY_KEY, HConstants.DEFAULT_DISTRIBUTED_LOG_REPLAY_CONFIG); @@ -791,6 +790,11 @@ } } + // Initialize the RegionServerCoprocessorHost now that our ephemeral + // node was created by reportForDuty, in case any coprocessors want + // to use ZooKeeper + this.rsHost = new RegionServerCoprocessorHost(this, this.conf); + if (!this.stopped && isHealthy()){ // start the snapshot handler, since the server is ready to run this.snapshotManager.start(); Looks like moving the initialization has left something uninitialized during the test because when the abort happens there is what looks like an unexpected NPE: 2014-01-21 23:16:22,279 FATAL [RS:0;vesta:49655] regionserver.HRegionServer(1733): ABORTING region server vesta.apache.org,49655,1390346181063: Unhandled: null java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1665) at org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:1761) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.abortRegionServer(MiniHBaseCluster.java:173) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$100(MiniHBaseCluster.java:107) at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$2.run(MiniHBaseCluster.java:166)
        Hide
        Andrew Purtell added a comment -
        Show
        Andrew Purtell added a comment - Ping Vandana Ayyalasomayajula
        Hide
        Vandana Ayyalasomayajula added a comment -

        Andrew Purtell Taking a look at the NPE. Sorry for the delay, I was traveling.

        Show
        Vandana Ayyalasomayajula added a comment - Andrew Purtell Taking a look at the NPE. Sorry for the delay, I was traveling.
        Hide
        Vandana Ayyalasomayajula added a comment -

        This patch should fix the NPE as the region server coprocessor host is not initialized in the constructor.

        Show
        Vandana Ayyalasomayajula added a comment - This patch should fix the NPE as the region server coprocessor host is not initialized in the constructor.
        Hide
        Andrew Purtell added a comment -

        This patch should fix the NPE as the region server coprocessor host is not initialized in the constructor.

        lgtm, applying addendum to trunk and 0.98.

        Show
        Andrew Purtell added a comment - This patch should fix the NPE as the region server coprocessor host is not initialized in the constructor. lgtm, applying addendum to trunk and 0.98.
        Hide
        stack added a comment -

        I need it in 0.96 too boss Andrew Purtell Thanks. (Thanks Vandana Ayyalasomayajula for fixing while traveling).

        Show
        stack added a comment - I need it in 0.96 too boss Andrew Purtell Thanks. (Thanks Vandana Ayyalasomayajula for fixing while traveling).
        Hide
        Andrew Purtell added a comment -

        ... and will commit fix to 0.96 also, thanks stack

        Show
        Andrew Purtell added a comment - ... and will commit fix to 0.96 also, thanks stack
        Hide
        Andrew Purtell added a comment -

        Committed addendum to trunk, 0.98, and 0.96. Patch applied everywhere with minor fuzz only, failing test completes successfully with no NPE reported in the log.

        Show
        Andrew Purtell added a comment - Committed addendum to trunk, 0.98, and 0.96. Patch applied everywhere with minor fuzz only, failing test completes successfully with no NPE reported in the log.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #4850 (See https://builds.apache.org/job/HBase-TRUNK/4850/)
        Amend HBASE-10338 Fix NPE if the server is terminated before the RegionServerCoprocessorHost is initialized (Vandana Ayyalasomayajula) (apurtell: rev 1560494)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4850 (See https://builds.apache.org/job/HBase-TRUNK/4850/ ) Amend HBASE-10338 Fix NPE if the server is terminated before the RegionServerCoprocessorHost is initialized (Vandana Ayyalasomayajula) (apurtell: rev 1560494) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.96 #268 (See https://builds.apache.org/job/hbase-0.96/268/)
        Amend HBASE-10338 Fix NPE if the server is terminated before the RegionServerCoprocessorHost is initialized (Vandana Ayyalasomayajula) (apurtell: rev 1560496)

        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.96 #268 (See https://builds.apache.org/job/hbase-0.96/268/ ) Amend HBASE-10338 Fix NPE if the server is terminated before the RegionServerCoprocessorHost is initialized (Vandana Ayyalasomayajula) (apurtell: rev 1560496) /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.96-hadoop2 #184 (See https://builds.apache.org/job/hbase-0.96-hadoop2/184/)
        Amend HBASE-10338 Fix NPE if the server is terminated before the RegionServerCoprocessorHost is initialized (Vandana Ayyalasomayajula) (apurtell: rev 1560496)

        • /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.96-hadoop2 #184 (See https://builds.apache.org/job/hbase-0.96-hadoop2/184/ ) Amend HBASE-10338 Fix NPE if the server is terminated before the RegionServerCoprocessorHost is initialized (Vandana Ayyalasomayajula) (apurtell: rev 1560496) /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.

          People

          • Assignee:
            Vandana Ayyalasomayajula
            Reporter:
            Vandana Ayyalasomayajula
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development