Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.5
    • Component/s: Client, IPC/RPC
    • Labels:
      None
    • Release Note:
      Hide
      Overriding the configured RPC engine separately for each RPC protocol (hbase.rpc.engine.<protocol class> configuration property) is no longer supported. Only a single RPC engine configuration (hbase.rpc.engine) is used.

      The HConnectionManager#deleteConnection(Configuration, boolean) and HConnectionManager#deleteAllConnections(boolean) methods are now deprecated, as the "stopProxy" parameter is no longer used.

      In some cases, where clients explicitly manage HConnection instance creation, the number of client RPC connections created may change. This issue changes the HConnection implementation to use it's own managed HBaseClient instance, with its own set of client RPC connections. So explicitly creating multiple HConnection instances (using HConnectionManager.createConnection()) will result in multiple HBaseClient instances. However, for the default behavior, using HTable with a single Configuration, there is no change.
      Show
      Overriding the configured RPC engine separately for each RPC protocol (hbase.rpc.engine.<protocol class> configuration property) is no longer supported. Only a single RPC engine configuration (hbase.rpc.engine) is used. The HConnectionManager#deleteConnection(Configuration, boolean) and HConnectionManager#deleteAllConnections(boolean) methods are now deprecated, as the "stopProxy" parameter is no longer used. In some cases, where clients explicitly manage HConnection instance creation, the number of client RPC connections created may change. This issue changes the HConnection implementation to use it's own managed HBaseClient instance, with its own set of client RPC connections. So explicitly creating multiple HConnection instances (using HConnectionManager.createConnection()) will result in multiple HBaseClient instances. However, for the default behavior, using HTable with a single Configuration, there is no change.

      Description

      Marking critical so it gets in.

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          9d 22h 13m 1 Gary Helmling 28/Jan/13 22:19
          Patch Available Patch Available Resolved Resolved
          2h 37m 1 Gary Helmling 29/Jan/13 00:56
          Resolved Resolved Closed Closed
          18d 3h 17m 1 Lars Hofhansl 16/Feb/13 04:14
          Lars Hofhansl made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gary Helmling made changes -
          Link This issue is related to HBASE-7772 [ HBASE-7772 ]
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/)
          HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723)

          Result = FAILURE
          garyh :
          Files :

          • /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java
          • /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
          • /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/ ) HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723) Result = FAILURE garyh : Files : /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security #102 (See https://builds.apache.org/job/HBase-0.94-security/102/)
          HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723)

          Result = SUCCESS
          garyh :
          Files :

          • /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java
          • /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
          • /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security #102 (See https://builds.apache.org/job/HBase-0.94-security/102/ ) HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723) Result = SUCCESS garyh : Files : /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Hide
          Lars Hofhansl added a comment -

          Thanks Gary!

          Show
          Lars Hofhansl added a comment - Thanks Gary!
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #798 (See https://builds.apache.org/job/HBase-0.94/798/)
          HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723)

          Result = SUCCESS
          garyh :
          Files :

          • /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java
          • /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
          • /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #798 (See https://builds.apache.org/job/HBase-0.94/798/ ) HBASE-7626 Backport client connection cleanup from HBASE-7460 (Revision 1439723) Result = SUCCESS garyh : Files : /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java /hbase/branches/0.94/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java /hbase/branches/0.94/src/main/resources/hbase-webapps/master/table.jsp /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestDelayedRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestPBOnWritableRpc.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/ipc/TestProtocolExtension.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java
          Gary Helmling made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Release Note Overriding the configured RPC engine separately for each RPC protocol (hbase.rpc.engine.<protocol class> configuration property) is no longer supported. Only a single RPC engine configuration (hbase.rpc.engine) is used.

          The HConnectionManager#deleteConnection(Configuration, boolean) and HConnectionManager#deleteAllConnections(boolean) methods are now deprecated, as the "stopProxy" parameter is no longer used.

          In some cases, where clients explicitly manage HConnection instance creation, the number of client RPC connections created may change. This issue changes the HConnection implementation to use it's own managed HBaseClient instance, with its own set of client RPC connections. So explicitly creating multiple HConnection instances (using HConnectionManager.createConnection()) will result in multiple HBaseClient instances. However, for the default behavior, using HTable with a single Configuration, there is no change.
          Resolution Fixed [ 1 ]
          Hide
          Gary Helmling added a comment -

          Committed to 0.94 branch. Thanks for reviews, Ted and Lars!

          Show
          Gary Helmling added a comment - Committed to 0.94 branch. Thanks for reviews, Ted and Lars!
          Hide
          Ted Yu added a comment -

          Hadoop QA only checks patches for trunk.

          Show
          Ted Yu added a comment - Hadoop QA only checks patches for trunk.
          Hide
          Gary Helmling added a comment -

          Is HadoopQA smart enough to apply 0.94 patches? If so, what do I need to do to make this happen?

          Show
          Gary Helmling added a comment - Is HadoopQA smart enough to apply 0.94 patches? If so, what do I need to do to make this happen?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12566832/HBASE-7626-2_0.94.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 33 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4218//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566832/HBASE-7626-2_0.94.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 33 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4218//console This message is automatically generated.
          Gary Helmling made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Gary Helmling made changes -
          Assignee Gary Helmling [ ghelmling ]
          Gary Helmling made changes -
          Attachment HBASE-7626-2_0.94.patch [ 12566832 ]
          Hide
          Gary Helmling added a comment -

          Attaching the current RB patch.

          I ran this through the full test suite with no failures.

          Show
          Gary Helmling added a comment - Attaching the current RB patch. I ran this through the full test suite with no failures.
          Hide
          Lars Hofhansl added a comment -

          Yeah me too +1. Risky, but needs to be done I think.

          Show
          Lars Hofhansl added a comment - Yeah me too +1. Risky, but needs to be done I think.
          Hide
          Ted Yu added a comment -

          +1 on latest patch.

          Show
          Ted Yu added a comment - +1 on latest patch.
          Hide
          Gary Helmling added a comment -

          Posted an initial patch for review: https://reviews.apache.org/r/9112/

          Show
          Gary Helmling added a comment - Posted an initial patch for review: https://reviews.apache.org/r/9112/
          Lars Hofhansl made changes -
          Priority Major [ 3 ] Critical [ 2 ]
          Description Marking critical so it gets in.
          Lars Hofhansl made changes -
          Field Original Value New Value
          Component/s Client [ 12312131 ]
          Component/s IPC/RPC [ 12312136 ]
          Lars Hofhansl created issue -

            People

            • Assignee:
              Gary Helmling
              Reporter:
              Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development