HBase
  1. HBase
  2. HBASE-4508

Backport HBASE-3777 to 0.90 branch

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.90.5
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      A new config parameter, hbase.connection.per.config, has been added.
      If set to true, there is no connection sharing. This is default setting.
      If set to false, connection sharing would be enabled so that fewer connections to zookeeper are used.
      Show
      A new config parameter, hbase.connection.per.config, has been added. If set to true, there is no connection sharing. This is default setting. If set to false, connection sharing would be enabled so that fewer connections to zookeeper are used.

      Description

      See discussion here: http://search-hadoop.com/m/MJBId1aazTR1/backporting+HBASE-3777+to+0.90&subj=backporting+HBASE+3777+to+0+90

      Rocketfuel has been running 0.90.3 with HBASE-3777 since its resolution.
      They have 10 RS nodes , 1 Master and 1 Zookeeper
      Live writes and reads but super heavy on reads. Cache hit is pretty high.
      The qps on one of their data centers is 50K.

      1. HBASE-4508.v5.patch
        76 kB
        Bright Fulton
      2. HBASE-4508.v4.patch
        75 kB
        Bright Fulton
      3. HBASE-4508.v4.git.patch
        78 kB
        Ted Yu
      4. HBASE-4508.v3.patch
        74 kB
        Bright Fulton
      5. HBASE-4508.v2.patch
        74 kB
        Bright Fulton
      6. HBASE-4508.v1.patch
        74 kB
        Bright Fulton

        Issue Links

          Activity

          Hide
          Bright Fulton added a comment -

          This is HBASE-3777.v8.patch backported to 0.90 at rev 1175062 and with the addition of the hbase.client.per.config.inst config param.

          Show
          Bright Fulton added a comment - This is HBASE-3777 .v8.patch backported to 0.90 at rev 1175062 and with the addition of the hbase.client.per.config.inst config param.
          Hide
          Bright Fulton added a comment -

          Renamed hbase.client.per.config.inst to hbase.connection.per.config based on feedback.

          Cumulative diff is now against rev 1178940.

          Show
          Bright Fulton added a comment - Renamed hbase.client.per.config.inst to hbase.connection.per.config based on feedback . Cumulative diff is now against rev 1178940.
          Hide
          Bright Fulton added a comment -

          Attached cumulative v3 patch which includes a one line change to hbase.connection.per.config functionality to work with existing HConnectionKey hash and equality. Fixes TestZooKeeper.testMultipleZK.

          Show
          Bright Fulton added a comment - Attached cumulative v3 patch which includes a one line change to hbase.connection.per.config functionality to work with existing HConnectionKey hash and equality. Fixes TestZooKeeper.testMultipleZK.
          Hide
          Bright Fulton added a comment -

          Attached v4: changes to TestHCM so that new and newly enabled tests there unset hbase.connection.per.config to test shared-connection behavior.

          Show
          Bright Fulton added a comment - Attached v4: changes to TestHCM so that new and newly enabled tests there unset hbase.connection.per.config to test shared-connection behavior.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2192/
          -----------------------------------------------------------

          Review request for Ted Yu.

          Summary
          -------

          A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property.

          This addresses bug HBASE-4508.
          https://issues.apache.org/jira/browse/HBASE-4508

          Diffs


          ./src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 1179055
          ./src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 1179055
          ./src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 1179055
          ./src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 1179055
          ./src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 1179055
          ./src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1179055
          ./src/main/resources/hbase-default.xml 1179055
          ./src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/util/HMerge.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/client/HTable.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/client/HConnection.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 1179055
          ./src/main/java/org/apache/hadoop/hbase/HConstants.java 1179055

          Diff: https://reviews.apache.org/r/2192/diff

          Testing
          -------

          Thanks,

          Bright

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2192/ ----------------------------------------------------------- Review request for Ted Yu. Summary ------- A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property. This addresses bug HBASE-4508 . https://issues.apache.org/jira/browse/HBASE-4508 Diffs ./src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 1179055 ./src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 1179055 ./src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 1179055 ./src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 1179055 ./src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 1179055 ./src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 1179055 ./src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1179055 ./src/main/resources/hbase-default.xml 1179055 ./src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 1179055 ./src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1179055 ./src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1179055 ./src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1179055 ./src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 1179055 ./src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 1179055 ./src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 1179055 ./src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 1179055 ./src/main/java/org/apache/hadoop/hbase/util/HMerge.java 1179055 ./src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java 1179055 ./src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 1179055 ./src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 1179055 ./src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java 1179055 ./src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java 1179055 ./src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 1179055 ./src/main/java/org/apache/hadoop/hbase/client/HTable.java 1179055 ./src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 1179055 ./src/main/java/org/apache/hadoop/hbase/client/HConnection.java 1179055 ./src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 1179055 ./src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 1179055 ./src/main/java/org/apache/hadoop/hbase/HConstants.java 1179055 Diff: https://reviews.apache.org/r/2192/diff Testing ------- Thanks, Bright
          Hide
          Ted Yu added a comment -

          +1 on patch v4.

          Show
          Ted Yu added a comment - +1 on patch v4.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2448/
          -----------------------------------------------------------

          Review request for hbase, Ted Yu and Bright Fulton.

          Summary
          -------

          A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property.

          This addresses bug HBASE-4508.
          https://issues.apache.org/jira/browse/HBASE-4508

          Diffs


          src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149
          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf
          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136
          src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554
          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583
          src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167
          src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9
          src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9
          src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333
          src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb
          src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e
          src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa
          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20
          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc
          src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33
          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b
          src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a
          src/main/resources/hbase-default.xml de2e91f
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f
          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3
          src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75
          src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8
          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6
          src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb

          Diff: https://reviews.apache.org/r/2448/diff

          Testing
          -------

          Thanks,

          Bright

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/ ----------------------------------------------------------- Review request for hbase, Ted Yu and Bright Fulton. Summary ------- A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property. This addresses bug HBASE-4508 . https://issues.apache.org/jira/browse/HBASE-4508 Diffs src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136 src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583 src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167 src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9 src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9 src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333 src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33 src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a src/main/resources/hbase-default.xml de2e91f src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75 src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb Diff: https://reviews.apache.org/r/2448/diff Testing ------- Thanks, Bright
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2448/
          -----------------------------------------------------------

          (Updated 2011-10-20 10:47:54.108738)

          Review request for hbase, Ted Yu and Bright Fulton.

          Summary
          -------

          A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property.

          This addresses bug HBASE-4508.
          https://issues.apache.org/jira/browse/HBASE-4508

          Diffs


          src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149
          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf
          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136
          src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554
          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583
          src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167
          src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9
          src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9
          src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333
          src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb
          src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e
          src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa
          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20
          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc
          src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33
          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b
          src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a
          src/main/resources/hbase-default.xml de2e91f
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f
          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3
          src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75
          src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8
          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6
          src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb

          Diff: https://reviews.apache.org/r/2448/diff

          Testing (updated)
          -------

          All tests pass.

          Thanks,

          Bright

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/ ----------------------------------------------------------- (Updated 2011-10-20 10:47:54.108738) Review request for hbase, Ted Yu and Bright Fulton. Summary ------- A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property. This addresses bug HBASE-4508 . https://issues.apache.org/jira/browse/HBASE-4508 Diffs src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136 src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583 src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167 src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9 src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9 src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333 src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33 src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a src/main/resources/hbase-default.xml de2e91f src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75 src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb Diff: https://reviews.apache.org/r/2448/diff Testing (updated) ------- All tests pass. Thanks, Bright
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2448/#review2705
          -----------------------------------------------------------

          Ship it!

          Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has.

          • Michael

          On 2011-10-20 10:47:54, Bright Fulton wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2448/

          -----------------------------------------------------------

          (Updated 2011-10-20 10:47:54)

          Review request for hbase, Ted Yu and Bright Fulton.

          Summary

          -------

          A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property.

          This addresses bug HBASE-4508.

          https://issues.apache.org/jira/browse/HBASE-4508

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149

          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136

          src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554

          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583

          src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167

          src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9

          src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9

          src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333

          src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb

          src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e

          src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa

          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20

          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc

          src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33

          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b

          src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a

          src/main/resources/hbase-default.xml de2e91f

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f

          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3

          src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75

          src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8

          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6

          src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb

          Diff: https://reviews.apache.org/r/2448/diff

          Testing

          -------

          All tests pass.

          Thanks,

          Bright

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/#review2705 ----------------------------------------------------------- Ship it! Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has. Michael On 2011-10-20 10:47:54, Bright Fulton wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/ ----------------------------------------------------------- (Updated 2011-10-20 10:47:54) Review request for hbase, Ted Yu and Bright Fulton. Summary ------- A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property. This addresses bug HBASE-4508 . https://issues.apache.org/jira/browse/HBASE-4508 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136 src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583 src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167 src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9 src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9 src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333 src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33 src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a src/main/resources/hbase-default.xml de2e91f src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75 src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb Diff: https://reviews.apache.org/r/2448/diff Testing ------- All tests pass. Thanks, Bright
          Hide
          Ted Yu added a comment -

          By compatibility I guess you mean not sharing connections.
          hbase.connection.per.config config property has default value of true, keeping the existing behavior in 0.90.4

          Show
          Ted Yu added a comment - By compatibility I guess you mean not sharing connections. hbase.connection.per.config config property has default value of true, keeping the existing behavior in 0.90.4
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-10-20 17:31:06, Michael Stack wrote:

          > Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has.

          TestHCM.testManyNewConnectionsDoesnotOOME did OOME once we introduced hbase.connection.per.config (default true) and before we set it false in that test.

          • Bright

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2448/#review2705
          -----------------------------------------------------------

          On 2011-10-20 10:47:54, Bright Fulton wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2448/

          -----------------------------------------------------------

          (Updated 2011-10-20 10:47:54)

          Review request for hbase, Ted Yu and Bright Fulton.

          Summary

          -------

          A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property.

          This addresses bug HBASE-4508.

          https://issues.apache.org/jira/browse/HBASE-4508

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149

          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136

          src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554

          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583

          src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167

          src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9

          src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9

          src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333

          src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb

          src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e

          src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa

          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20

          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc

          src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33

          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b

          src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a

          src/main/resources/hbase-default.xml de2e91f

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f

          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3

          src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75

          src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8

          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6

          src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb

          Diff: https://reviews.apache.org/r/2448/diff

          Testing

          -------

          All tests pass.

          Thanks,

          Bright

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-10-20 17:31:06, Michael Stack wrote: > Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has. TestHCM.testManyNewConnectionsDoesnotOOME did OOME once we introduced hbase.connection.per.config (default true) and before we set it false in that test. Bright ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/#review2705 ----------------------------------------------------------- On 2011-10-20 10:47:54, Bright Fulton wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/ ----------------------------------------------------------- (Updated 2011-10-20 10:47:54) Review request for hbase, Ted Yu and Bright Fulton. Summary ------- A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property. This addresses bug HBASE-4508 . https://issues.apache.org/jira/browse/HBASE-4508 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136 src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583 src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167 src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9 src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9 src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333 src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33 src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a src/main/resources/hbase-default.xml de2e91f src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75 src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb Diff: https://reviews.apache.org/r/2448/diff Testing ------- All tests pass. Thanks, Bright
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-10-20 17:31:06, Michael Stack wrote:

          > Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has.

          Bright Fulton wrote:

          TestHCM.testManyNewConnectionsDoesnotOOME did OOME once we introduced hbase.connection.per.config (default true) and before we set it false in that test.

          I'm not sure what your point is Bright (Thanks).

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2448/#review2705
          -----------------------------------------------------------

          On 2011-10-20 10:47:54, Bright Fulton wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2448/

          -----------------------------------------------------------

          (Updated 2011-10-20 10:47:54)

          Review request for hbase, Ted Yu and Bright Fulton.

          Summary

          -------

          A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property.

          This addresses bug HBASE-4508.

          https://issues.apache.org/jira/browse/HBASE-4508

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149

          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136

          src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554

          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583

          src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167

          src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9

          src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9

          src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333

          src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb

          src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e

          src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa

          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20

          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc

          src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33

          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b

          src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a

          src/main/resources/hbase-default.xml de2e91f

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f

          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3

          src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75

          src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8

          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6

          src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb

          Diff: https://reviews.apache.org/r/2448/diff

          Testing

          -------

          All tests pass.

          Thanks,

          Bright

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-10-20 17:31:06, Michael Stack wrote: > Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has. Bright Fulton wrote: TestHCM.testManyNewConnectionsDoesnotOOME did OOME once we introduced hbase.connection.per.config (default true) and before we set it false in that test. I'm not sure what your point is Bright (Thanks). Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/#review2705 ----------------------------------------------------------- On 2011-10-20 10:47:54, Bright Fulton wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/ ----------------------------------------------------------- (Updated 2011-10-20 10:47:54) Review request for hbase, Ted Yu and Bright Fulton. Summary ------- A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property. This addresses bug HBASE-4508 . https://issues.apache.org/jira/browse/HBASE-4508 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136 src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583 src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167 src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9 src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9 src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333 src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33 src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a src/main/resources/hbase-default.xml de2e91f src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75 src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb Diff: https://reviews.apache.org/r/2448/diff Testing ------- All tests pass. Thanks, Bright
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-10-20 17:31:06, Michael Stack wrote:

          > Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has.

          Bright Fulton wrote:

          TestHCM.testManyNewConnectionsDoesnotOOME did OOME once we introduced hbase.connection.per.config (default true) and before we set it false in that test.

          Michael Stack wrote:

          I'm not sure what your point is Bright (Thanks).

          That test failing with the default config was a small demonstration of compatibility with prior behavior.

          • Bright

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2448/#review2705
          -----------------------------------------------------------

          On 2011-10-20 10:47:54, Bright Fulton wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2448/

          -----------------------------------------------------------

          (Updated 2011-10-20 10:47:54)

          Review request for hbase, Ted Yu and Bright Fulton.

          Summary

          -------

          A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property.

          This addresses bug HBASE-4508.

          https://issues.apache.org/jira/browse/HBASE-4508

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149

          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136

          src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554

          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583

          src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167

          src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9

          src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9

          src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333

          src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb

          src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e

          src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa

          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20

          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc

          src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33

          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b

          src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a

          src/main/resources/hbase-default.xml de2e91f

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f

          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3

          src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75

          src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8

          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6

          src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb

          Diff: https://reviews.apache.org/r/2448/diff

          Testing

          -------

          All tests pass.

          Thanks,

          Bright

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-10-20 17:31:06, Michael Stack wrote: > Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has. Bright Fulton wrote: TestHCM.testManyNewConnectionsDoesnotOOME did OOME once we introduced hbase.connection.per.config (default true) and before we set it false in that test. Michael Stack wrote: I'm not sure what your point is Bright (Thanks). That test failing with the default config was a small demonstration of compatibility with prior behavior. Bright ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/#review2705 ----------------------------------------------------------- On 2011-10-20 10:47:54, Bright Fulton wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/ ----------------------------------------------------------- (Updated 2011-10-20 10:47:54) Review request for hbase, Ted Yu and Bright Fulton. Summary ------- A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property. This addresses bug HBASE-4508 . https://issues.apache.org/jira/browse/HBASE-4508 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136 src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583 src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167 src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9 src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9 src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333 src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33 src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a src/main/resources/hbase-default.xml de2e91f src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75 src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb Diff: https://reviews.apache.org/r/2448/diff Testing ------- All tests pass. Thanks, Bright
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-10-20 17:31:06, Michael Stack wrote:

          > Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has.

          Bright Fulton wrote:

          TestHCM.testManyNewConnectionsDoesnotOOME did OOME once we introduced hbase.connection.per.config (default true) and before we set it false in that test.

          Michael Stack wrote:

          I'm not sure what your point is Bright (Thanks).

          Bright Fulton wrote:

          That test failing with the default config was a small demonstration of compatibility with prior behavior.

          Thanks. Now I understand.

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2448/#review2705
          -----------------------------------------------------------

          On 2011-10-20 10:47:54, Bright Fulton wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2448/

          -----------------------------------------------------------

          (Updated 2011-10-20 10:47:54)

          Review request for hbase, Ted Yu and Bright Fulton.

          Summary

          -------

          A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property.

          This addresses bug HBASE-4508.

          https://issues.apache.org/jira/browse/HBASE-4508

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149

          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136

          src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554

          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583

          src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167

          src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9

          src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9

          src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333

          src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb

          src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e

          src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa

          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20

          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc

          src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33

          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b

          src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a

          src/main/resources/hbase-default.xml de2e91f

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f

          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3

          src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75

          src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8

          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6

          src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb

          Diff: https://reviews.apache.org/r/2448/diff

          Testing

          -------

          All tests pass.

          Thanks,

          Bright

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-10-20 17:31:06, Michael Stack wrote: > Patch looks good to me. Good stuff. Before we commit, has anyone tested it doesn't break compatibility? It looks safe but its a big change so I think it needs testing. I'll do it if no one else has. Bright Fulton wrote: TestHCM.testManyNewConnectionsDoesnotOOME did OOME once we introduced hbase.connection.per.config (default true) and before we set it false in that test. Michael Stack wrote: I'm not sure what your point is Bright (Thanks). Bright Fulton wrote: That test failing with the default config was a small demonstration of compatibility with prior behavior. Thanks. Now I understand. Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/#review2705 ----------------------------------------------------------- On 2011-10-20 10:47:54, Bright Fulton wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2448/ ----------------------------------------------------------- (Updated 2011-10-20 10:47:54) Review request for hbase, Ted Yu and Bright Fulton. Summary ------- A backport of HBASE-3777 to the 0.90 branch, preserving 0.90 connection-per-config behavior by introducing the (default true) hbase.connection.per.config config property. This addresses bug HBASE-4508 . https://issues.apache.org/jira/browse/HBASE-4508 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 611b149 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java 64cabdf src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java aabf136 src/main/java/org/apache/hadoop/hbase/client/HConnection.java ed2f554 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java e6d1583 src/main/java/org/apache/hadoop/hbase/client/HTable.java cd5f167 src/main/java/org/apache/hadoop/hbase/client/HTablePool.java 78cb3d9 src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java cfd86c9 src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java d76e333 src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java 80284bb src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java 686813e src/main/java/org/apache/hadoop/hbase/mapreduce/replication/VerifyReplication.java ed88bfa src/main/java/org/apache/hadoop/hbase/master/HMaster.java 88a5c20 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 923f891 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 72ba8dc src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java 133da33 src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 85cbb9e src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java b04aab6 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06ac90b src/main/java/org/apache/hadoop/hbase/util/HMerge.java d727aa5 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java ec9f029 src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java f277e6a src/main/resources/hbase-default.xml de2e91f src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java 910d68f src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/client/TestHCM.java 5d71d75 src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java 624f4a8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 8992dbb Diff: https://reviews.apache.org/r/2448/diff Testing ------- All tests pass. Thanks, Bright
          Hide
          Ted Yu added a comment -

          Latest patch from review board.

          Show
          Ted Yu added a comment - Latest patch from review board.
          Hide
          Bright Fulton added a comment -

          Remove TestHCM.getValidKeyCount() from patch to avoid ConcurrentModificationException:

          testConnectionUniqueness(org.apache.hadoop.hbase.client.TestHCM) Time elapsed: 1.978 sec <<< ERROR!
          java.util.ConcurrentModificationException
          at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:373)
          at java.util.LinkedHashMap$KeyIterator.next(LinkedHashMap.java:384)
          at java.util.AbstractCollection.toArray(AbstractCollection.java:124)
          at java.util.ArrayList.<init>(ArrayList.java:131)
          at org.apache.hadoop.hbase.client.TestHCM.getValidKeyCount(TestHCM.java:136)
          at org.apache.hadoop.hbase.client.TestHCM.testConnectionUniqueness(TestHCM.java:222)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

          Based on revision 1188010.

          Show
          Bright Fulton added a comment - Remove TestHCM.getValidKeyCount() from patch to avoid ConcurrentModificationException: testConnectionUniqueness(org.apache.hadoop.hbase.client.TestHCM) Time elapsed: 1.978 sec <<< ERROR! java.util.ConcurrentModificationException at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:373) at java.util.LinkedHashMap$KeyIterator.next(LinkedHashMap.java:384) at java.util.AbstractCollection.toArray(AbstractCollection.java:124) at java.util.ArrayList.<init>(ArrayList.java:131) at org.apache.hadoop.hbase.client.TestHCM.getValidKeyCount(TestHCM.java:136) at org.apache.hadoop.hbase.client.TestHCM.testConnectionUniqueness(TestHCM.java:222) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Based on revision 1188010.
          Hide
          Ted Yu added a comment -

          Integrated HBASE-4508.v5.patch to 0.90

          Thanks for the patch Bright.

          Thanks for the validation Shrijeet.

          Show
          Ted Yu added a comment - Integrated HBASE-4508 .v5.patch to 0.90 Thanks for the patch Bright. Thanks for the validation Shrijeet.
          Hide
          Ted Yu added a comment -

          From Shrijeet:

          1. Rolling restart with 0.90 trunk + patch jar : PASSED
          2. Verify that 0.90 trunk jar leaks connection with HTable pool size set to 10 , hbase.zookeeper.property.maxClientCnxns set to 20 and launching 10 client threads : PASSED
          3. Verify that 0.90 trunk jar + patch does not leak connection with HTable pool size set to 10 , hbase.zookeeper.property.maxClientCnxns set to 20 and launching 10 client threads : PASSED
          4. Verify that 0.90 trunk jar + patch still holds up when HTable pool size set to 10 , hbase.zookeeper.property.
          maxClientCnxns set to 5 and launching 100 client threads : PASSED

          The established connections in my final test was 2. This I verified using both netstat and zookeeper dump in UI.

          One caveat: remember to set hbase.connection.per.config false in order to use connection sharing.

          5. hbase.connection.per.config=true , max connections to 300 and bumping ulimit from 1024 to 2048, client threads 100 : PASSED

          Show
          Ted Yu added a comment - From Shrijeet: 1. Rolling restart with 0.90 trunk + patch jar : PASSED 2. Verify that 0.90 trunk jar leaks connection with HTable pool size set to 10 , hbase.zookeeper.property.maxClientCnxns set to 20 and launching 10 client threads : PASSED 3. Verify that 0.90 trunk jar + patch does not leak connection with HTable pool size set to 10 , hbase.zookeeper.property.maxClientCnxns set to 20 and launching 10 client threads : PASSED 4. Verify that 0.90 trunk jar + patch still holds up when HTable pool size set to 10 , hbase.zookeeper.property. maxClientCnxns set to 5 and launching 100 client threads : PASSED The established connections in my final test was 2. This I verified using both netstat and zookeeper dump in UI. One caveat: remember to set hbase.connection.per.config false in order to use connection sharing. 5. hbase.connection.per.config=true , max connections to 300 and bumping ulimit from 1024 to 2048, client threads 100 : PASSED
          Hide
          stack added a comment -

          I just tried this patch against the tip of 0.90. On a cluster loaded with regions, I brought down one server, put in place a jar with this patch in place and then brought it back up moving regions back on to the server. A client that was not patched was able to talk to this server. It looks to me like this patch is safe to backport.

          Show
          stack added a comment - I just tried this patch against the tip of 0.90. On a cluster loaded with regions, I brought down one server, put in place a jar with this patch in place and then brought it back up moving regions back on to the server. A client that was not patched was able to talk to this server. It looks to me like this patch is safe to backport.

            People

            • Assignee:
              Bright Fulton
              Reporter:
              Ted Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development