Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.5
    • Component/s: Client, IPC/RPC
    • Labels:
      None
    • Release Note:
      Hide
      Overriding the configured RPC engine separately for each RPC protocol (hbase.rpc.engine.<protocol class> configuration property) is no longer supported. Only a single RPC engine configuration (hbase.rpc.engine) is used.

      The HConnectionManager#deleteConnection(Configuration, boolean) and HConnectionManager#deleteAllConnections(boolean) methods are now deprecated, as the "stopProxy" parameter is no longer used.

      In some cases, where clients explicitly manage HConnection instance creation, the number of client RPC connections created may change. This issue changes the HConnection implementation to use it's own managed HBaseClient instance, with its own set of client RPC connections. So explicitly creating multiple HConnection instances (using HConnectionManager.createConnection()) will result in multiple HBaseClient instances. However, for the default behavior, using HTable with a single Configuration, there is no change.
      Show
      Overriding the configured RPC engine separately for each RPC protocol (hbase.rpc.engine.<protocol class> configuration property) is no longer supported. Only a single RPC engine configuration (hbase.rpc.engine) is used. The HConnectionManager#deleteConnection(Configuration, boolean) and HConnectionManager#deleteAllConnections(boolean) methods are now deprecated, as the "stopProxy" parameter is no longer used. In some cases, where clients explicitly manage HConnection instance creation, the number of client RPC connections created may change. This issue changes the HConnection implementation to use it's own managed HBaseClient instance, with its own set of client RPC connections. So explicitly creating multiple HConnection instances (using HConnectionManager.createConnection()) will result in multiple HBaseClient instances. However, for the default behavior, using HTable with a single Configuration, there is no change.

      Description

      Marking critical so it gets in.

        Issue Links

          Activity

          Hide
          Gary Helmling added a comment -

          Posted an initial patch for review: https://reviews.apache.org/r/9112/

          Show
          Gary Helmling added a comment - Posted an initial patch for review: https://reviews.apache.org/r/9112/
          Hide
          Ted Yu added a comment -

          +1 on latest patch.

          Show
          Ted Yu added a comment - +1 on latest patch.
          Hide
          Lars Hofhansl added a comment -

          Yeah me too +1. Risky, but needs to be done I think.

          Show
          Lars Hofhansl added a comment - Yeah me too +1. Risky, but needs to be done I think.
          Hide
          Gary Helmling added a comment -

          Attaching the current RB patch.

          I ran this through the full test suite with no failures.

          Show
          Gary Helmling added a comment - Attaching the current RB patch. I ran this through the full test suite with no failures.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12566832/HBASE-7626-2_0.94.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 33 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4218//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566832/HBASE-7626-2_0.94.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 33 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4218//console This message is automatically generated.
          Hide
          Gary Helmling added a comment -

          Is HadoopQA smart enough to apply 0.94 patches? If so, what do I need to do to make this happen?

          Show
          Gary Helmling added a comment - Is HadoopQA smart enough to apply 0.94 patches? If so, what do I need to do to make this happen?
          Hide
          Ted Yu added a comment -

          Hadoop QA only checks patches for trunk.

          Show
          Ted Yu added a comment - Hadoop QA only checks patches for trunk.
          Hide
          Gary Helmling added a comment -

          Committed to 0.94 branch. Thanks for reviews, Ted and Lars!

          Show
          Gary Helmling added a comment - Committed to 0.94 branch. Thanks for reviews, Ted and Lars!
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #798 (See https://builds.apache.org/job/HBase-0.94/798/)
          HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723)

          Result = SUCCESS
          garyh :
          Files :

          • /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java
          • /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
          • /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #798 (See https://builds.apache.org/job/HBase-0.94/798/ ) HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723) Result = SUCCESS garyh : Files : /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Hide
          Lars Hofhansl added a comment -

          Thanks Gary!

          Show
          Lars Hofhansl added a comment - Thanks Gary!
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security #102 (See https://builds.apache.org/job/HBase-0.94-security/102/)
          HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723)

          Result = SUCCESS
          garyh :
          Files :

          • /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java
          • /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
          • /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security #102 (See https://builds.apache.org/job/HBase-0.94-security/102/ ) HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723) Result = SUCCESS garyh : Files : /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/)
          HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723)

          Result = FAILURE
          garyh :
          Files :

          • /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java
          • /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
          • /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/ ) HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723) Result = FAILURE garyh : Files : /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java

            People

            • Assignee:
              Gary Helmling
              Reporter:
              Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development