HBase
  1. HBase
  2. HBASE-1502

Remove need for heartbeats in HBase

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.92.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Deprecates what were once considered fundamental classes, HServerInfo and HServerAddress. The former has been made irrelevant because its constituent info -- server name, server load -- is now passed otherwise. HServerAddress, a class that on deserialization, did a resolve (it created a new InetSocketAddress on deserialization) was just a bad idea from the get-go. Unfortunately, HSI and HSA were all over our codebase like a cancer. Hopefully the patient survives their purging.
      Show
      Deprecates what were once considered fundamental classes, HServerInfo and HServerAddress. The former has been made irrelevant because its constituent info -- server name, server load -- is now passed otherwise. HServerAddress, a class that on deserialization, did a resolve (it created a new InetSocketAddress on deserialization) was just a bad idea from the get-go. Unfortunately, HSI and HSA were all over our codebase like a cancer. Hopefully the patient survives their purging.

      Description

      HBase currently uses heartbeats between region servers and the master, piggybacking information on them when it can. This issue is to investigate if we can get rid of the need for those using ZooKeeper events.

      1. 1502-v7.txt
        426 kB
        stack
      2. 1502-v6.txt
        427 kB
        stack
      3. 1502-v5.txt
        427 kB
        stack
      4. 1502-4.txt
        332 kB
        stack
      5. 1502-v2.txt
        126 kB
        stack
      6. 1502.txt
        39 kB
        stack

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #1976 (See https://builds.apache.org/job/HBase-TRUNK/1976/)

          Show
          Hudson added a comment - Integrated in HBase-TRUNK #1976 (See https://builds.apache.org/job/HBase-TRUNK/1976/ )
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #1909 (See https://builds.apache.org/hudson/job/HBase-TRUNK/1909/)

          Show
          Hudson added a comment - Integrated in HBase-TRUNK #1909 (See https://builds.apache.org/hudson/job/HBase-TRUNK/1909/ )
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #1888 (See https://builds.apache.org/hudson/job/HBase-TRUNK/1888/)

          Show
          Hudson added a comment - Integrated in HBase-TRUNK #1888 (See https://builds.apache.org/hudson/job/HBase-TRUNK/1888/ )
          Hide
          stack added a comment -

          Committed to trunk (Sorry for test breakage – I'm on it. Sorry for making so big a patch. Won't do it again).

          Show
          stack added a comment - Committed to trunk (Sorry for test breakage – I'm on it. Sorry for making so big a patch. Won't do it again).
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-04-27 00:17:30, Jonathan Gray wrote:

          > src/main/java/org/apache/hadoop/hbase/ClusterStatus.java, line 113

          > <https://reviews.apache.org/r/647/diff/4/?file=17400#file17400line113>

          >

          > Can just use this.liveServers.values() for here and below?

          value is HServerLoad (Not a List of HRegionInfo). We want count of regions.

          I did remove the dup iteration having getAverageLoad instead call getRegionsCount.

          On 2011-04-27 00:17:30, Jonathan Gray wrote:

          > src/main/java/org/apache/hadoop/hbase/HServerInfo.java, line 42

          > <https://reviews.apache.org/r/647/diff/4/?file=17405#file17405line42>

          >

          > i see webuiport below, does this TODO still apply?

          It does.

          HSI is deprecated and after this patch goes in, there is no means of the RS telling Master of its webui port – in master we'll rely on configs. In rare case where webui port changes – it can if occupied – then master will be off.

          Will address elsewhere. Will write json to the RS ephemeral node with 'metadata' about the RS that will include webui port but likely other stuff that could be factored doing loading calculations, etc. (it used to hold a HSA but thats been bumped in this patch). Will file an issue after this patch goes in.

          On 2011-04-27 00:17:30, Jonathan Gray wrote:

          > src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java, line 83

          > <https://reviews.apache.org/r/647/diff/4/?file=17408#file17408line83>

          >

          > why String and not ServerName? because master has no startcode? (i see use of ServerName for master above tho)

          I changed it (This method is not used anywhere seemingly).

          On 2011-04-27 00:17:30, Jonathan Gray wrote:

          > src/main/java/org/apache/hadoop/hbase/HServerAddress.java, line 42

          > <https://reviews.apache.org/r/647/diff/4/?file=17404#file17404line42>

          >

          > Where is this actually used now? Should point it out here so it's clear and so that when it goes away we know we can get rid of this.

          Will do. Had a change of heart last night; previous was going to just 'sort-of' deprecate HSA but on way home, after thinking on it, HSA is just never a good idea so will outright deprecate it.

          On 2011-04-27 00:17:30, Jonathan Gray wrote:

          > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 792

          > <https://reviews.apache.org/r/647/diff/4/?file=17416#file17416line792>

          >

          > this is because HSA actually makes a connection or does the lookup?

          This I'll purge. I'll use the getConnection(ISA) that this patch adds instead so no reference to HSA in HBaseAdmin (except deprecated uses)

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/647/#review568
          -----------------------------------------------------------

          On 2011-04-26 23:50:23, Michael Stack wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/647/

          -----------------------------------------------------------

          (Updated 2011-04-26 23:50:23)

          Review request for hbase.

          Summary

          -------

          This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to

          send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie.

          RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed.

          We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree.

          Removed HMsg.

          HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation).

          Otherwise, we don't use it internally anymore.

          HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now.

          Fixed up regionserver and master startup so they now look the same.

          New tests

          Cruft cleanup.

          This addresses bug hbase-1502.

          https://issues.apache.org/jira/browse/hbase-1502

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef

          src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639

          src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00

          src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8

          src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472

          src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94

          src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053

          src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab

          src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742

          src/main/java/org/apache/hadoop/hbase/Server.java df396fa

          src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67

          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179

          src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031

          src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5

          src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a

          src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725

          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a

          src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56

          src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4

          src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f

          src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342

          src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6

          src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463

          src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4

          src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3

          src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5

          src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 2273e55

          src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345

          src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30

          src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4

          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba

          src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc

          src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e

          src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162

          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9

          src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java dada818

          src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a

          src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7

          src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2

          src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11

          src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e

          src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java fe807f3

          src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6

          src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992

          src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb

          src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d

          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42

          src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14

          src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28

          src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27

          src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484

          src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377

          src/main/resources/hbase-webapps/master/master.jsp eeabc0a

          src/main/resources/hbase-webapps/master/table.jsp 6fcf16f

          src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205

          src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe

          src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc

          src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959

          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3

          src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e

          src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa

          src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03

          src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941

          src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8

          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6

          src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d

          src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1

          src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af

          src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53

          src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb

          src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f

          src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0

          src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b

          src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6

          src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e

          src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e

          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 9dd71a3

          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9

          src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327

          src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8

          src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082

          Diff: https://reviews.apache.org/r/647/diff

          Testing

          -------

          Most tests pass. I have yet to run it on a cluster. Doing that now.

          Thanks,

          Michael

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-04-27 00:17:30, Jonathan Gray wrote: > src/main/java/org/apache/hadoop/hbase/ClusterStatus.java, line 113 > < https://reviews.apache.org/r/647/diff/4/?file=17400#file17400line113 > > > Can just use this.liveServers.values() for here and below? value is HServerLoad (Not a List of HRegionInfo). We want count of regions. I did remove the dup iteration having getAverageLoad instead call getRegionsCount. On 2011-04-27 00:17:30, Jonathan Gray wrote: > src/main/java/org/apache/hadoop/hbase/HServerInfo.java, line 42 > < https://reviews.apache.org/r/647/diff/4/?file=17405#file17405line42 > > > i see webuiport below, does this TODO still apply? It does. HSI is deprecated and after this patch goes in, there is no means of the RS telling Master of its webui port – in master we'll rely on configs. In rare case where webui port changes – it can if occupied – then master will be off. Will address elsewhere. Will write json to the RS ephemeral node with 'metadata' about the RS that will include webui port but likely other stuff that could be factored doing loading calculations, etc. (it used to hold a HSA but thats been bumped in this patch). Will file an issue after this patch goes in. On 2011-04-27 00:17:30, Jonathan Gray wrote: > src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java, line 83 > < https://reviews.apache.org/r/647/diff/4/?file=17408#file17408line83 > > > why String and not ServerName? because master has no startcode? (i see use of ServerName for master above tho) I changed it (This method is not used anywhere seemingly). On 2011-04-27 00:17:30, Jonathan Gray wrote: > src/main/java/org/apache/hadoop/hbase/HServerAddress.java, line 42 > < https://reviews.apache.org/r/647/diff/4/?file=17404#file17404line42 > > > Where is this actually used now? Should point it out here so it's clear and so that when it goes away we know we can get rid of this. Will do. Had a change of heart last night; previous was going to just 'sort-of' deprecate HSA but on way home, after thinking on it, HSA is just never a good idea so will outright deprecate it. On 2011-04-27 00:17:30, Jonathan Gray wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 792 > < https://reviews.apache.org/r/647/diff/4/?file=17416#file17416line792 > > > this is because HSA actually makes a connection or does the lookup? This I'll purge. I'll use the getConnection(ISA) that this patch adds instead so no reference to HSA in HBaseAdmin (except deprecated uses) Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/#review568 ----------------------------------------------------------- On 2011-04-26 23:50:23, Michael Stack wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/ ----------------------------------------------------------- (Updated 2011-04-26 23:50:23) Review request for hbase. Summary ------- This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie. RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed. We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree. Removed HMsg. HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation). Otherwise, we don't use it internally anymore. HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now. Fixed up regionserver and master startup so they now look the same. New tests Cruft cleanup. This addresses bug hbase-1502. https://issues.apache.org/jira/browse/hbase-1502 Diffs ----- src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639 src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00 src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8 src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472 src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94 src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053 src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742 src/main/java/org/apache/hadoop/hbase/Server.java df396fa src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179 src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031 src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5 src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56 src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4 src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342 src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6 src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463 src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4 src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3 src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 2273e55 src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345 src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30 src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4 src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9 src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java dada818 src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7 src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2 src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11 src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java fe807f3 src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6 src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992 src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42 src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14 src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28 src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27 src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484 src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377 src/main/resources/hbase-webapps/master/master.jsp eeabc0a src/main/resources/hbase-webapps/master/table.jsp 6fcf16f src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205 src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959 src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03 src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1 src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53 src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0 src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6 src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 9dd71a3 src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9 src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327 src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082 Diff: https://reviews.apache.org/r/647/diff Testing ------- Most tests pass. I have yet to run it on a cluster. Doing that now. Thanks, Michael
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/647/#review585
          -----------------------------------------------------------

          Thanks for review jg

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          <https://reviews.apache.org/r/647/#comment1226>

          This I've been changing removing HSA. Instead I'm passing an ISA (this patch adds a getConnection(isa) method).

          • Michael

          On 2011-04-26 23:50:23, Michael Stack wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/647/

          -----------------------------------------------------------

          (Updated 2011-04-26 23:50:23)

          Review request for hbase.

          Summary

          -------

          This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to

          send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie.

          RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed.

          We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree.

          Removed HMsg.

          HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation).

          Otherwise, we don't use it internally anymore.

          HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now.

          Fixed up regionserver and master startup so they now look the same.

          New tests

          Cruft cleanup.

          This addresses bug hbase-1502.

          https://issues.apache.org/jira/browse/hbase-1502

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef

          src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639

          src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00

          src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8

          src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472

          src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94

          src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053

          src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab

          src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742

          src/main/java/org/apache/hadoop/hbase/Server.java df396fa

          src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67

          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179

          src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031

          src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5

          src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a

          src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725

          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a

          src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56

          src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4

          src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f

          src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342

          src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6

          src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463

          src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4

          src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3

          src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5

          src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 2273e55

          src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345

          src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30

          src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4

          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba

          src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc

          src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e

          src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162

          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9

          src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java dada818

          src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a

          src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7

          src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2

          src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11

          src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e

          src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java fe807f3

          src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6

          src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992

          src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb

          src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d

          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42

          src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14

          src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28

          src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27

          src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484

          src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377

          src/main/resources/hbase-webapps/master/master.jsp eeabc0a

          src/main/resources/hbase-webapps/master/table.jsp 6fcf16f

          src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205

          src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe

          src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc

          src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959

          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3

          src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e

          src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa

          src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03

          src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941

          src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8

          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6

          src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d

          src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1

          src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af

          src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53

          src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb

          src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f

          src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0

          src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b

          src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6

          src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e

          src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e

          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 9dd71a3

          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9

          src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327

          src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8

          src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082

          Diff: https://reviews.apache.org/r/647/diff

          Testing

          -------

          Most tests pass. I have yet to run it on a cluster. Doing that now.

          Thanks,

          Michael

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/#review585 ----------------------------------------------------------- Thanks for review jg src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java < https://reviews.apache.org/r/647/#comment1226 > This I've been changing removing HSA. Instead I'm passing an ISA (this patch adds a getConnection(isa) method). Michael On 2011-04-26 23:50:23, Michael Stack wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/ ----------------------------------------------------------- (Updated 2011-04-26 23:50:23) Review request for hbase. Summary ------- This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie. RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed. We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree. Removed HMsg. HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation). Otherwise, we don't use it internally anymore. HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now. Fixed up regionserver and master startup so they now look the same. New tests Cruft cleanup. This addresses bug hbase-1502. https://issues.apache.org/jira/browse/hbase-1502 Diffs ----- src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639 src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00 src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8 src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472 src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94 src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053 src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742 src/main/java/org/apache/hadoop/hbase/Server.java df396fa src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179 src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031 src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5 src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56 src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4 src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342 src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6 src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463 src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4 src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3 src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 2273e55 src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345 src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30 src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4 src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9 src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java dada818 src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7 src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2 src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11 src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java fe807f3 src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6 src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992 src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42 src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14 src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28 src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27 src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484 src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377 src/main/resources/hbase-webapps/master/master.jsp eeabc0a src/main/resources/hbase-webapps/master/table.jsp 6fcf16f src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205 src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959 src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03 src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1 src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53 src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0 src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6 src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 9dd71a3 src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9 src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327 src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082 Diff: https://reviews.apache.org/r/647/diff Testing ------- Most tests pass. I have yet to run it on a cluster. Doing that now. Thanks, Michael
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/647/#review568
          -----------------------------------------------------------

          Ship it!

          woohoo! glad HMsg is dead!

          src/main/java/org/apache/hadoop/hbase/ClusterStatus.java
          <https://reviews.apache.org/r/647/#comment1206>

          Can just use this.liveServers.values() for here and below?

          src/main/java/org/apache/hadoop/hbase/HServerAddress.java
          <https://reviews.apache.org/r/647/#comment1207>

          Where is this actually used now? Should point it out here so it's clear and so that when it goes away we know we can get rid of this.

          src/main/java/org/apache/hadoop/hbase/HServerInfo.java
          <https://reviews.apache.org/r/647/#comment1208>

          i see webuiport below, does this TODO still apply?

          src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java
          <https://reviews.apache.org/r/647/#comment1209>

          why String and not ServerName? because master has no startcode? (i see use of ServerName for master above tho)

          src/main/java/org/apache/hadoop/hbase/ServerName.java
          <https://reviews.apache.org/r/647/#comment1210>

          awesome that this is tucked away in here now

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
          <https://reviews.apache.org/r/647/#comment1211>

          this is because HSA actually makes a connection or does the lookup?

          • Jonathan

          On 2011-04-26 23:50:23, Michael Stack wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/647/

          -----------------------------------------------------------

          (Updated 2011-04-26 23:50:23)

          Review request for hbase.

          Summary

          -------

          This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to

          send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie.

          RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed.

          We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree.

          Removed HMsg.

          HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation).

          Otherwise, we don't use it internally anymore.

          HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now.

          Fixed up regionserver and master startup so they now look the same.

          New tests

          Cruft cleanup.

          This addresses bug hbase-1502.

          https://issues.apache.org/jira/browse/hbase-1502

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef

          src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639

          src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00

          src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8

          src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472

          src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94

          src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053

          src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab

          src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742

          src/main/java/org/apache/hadoop/hbase/Server.java df396fa

          src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67

          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179

          src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031

          src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5

          src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5

          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a

          src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725

          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a

          src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56

          src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024

          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4

          src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f

          src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342

          src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6

          src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463

          src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4

          src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3

          src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5

          src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 2273e55

          src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345

          src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30

          src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4

          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba

          src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc

          src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e

          src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162

          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9

          src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java dada818

          src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a

          src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7

          src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2

          src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702

          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11

          src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e

          src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java fe807f3

          src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6

          src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992

          src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb

          src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d

          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42

          src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14

          src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34

          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28

          src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27

          src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484

          src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6

          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377

          src/main/resources/hbase-webapps/master/master.jsp eeabc0a

          src/main/resources/hbase-webapps/master/table.jsp 6fcf16f

          src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205

          src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe

          src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc

          src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e

          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959

          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3

          src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e

          src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa

          src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03

          src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941

          src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8

          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6

          src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d

          src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1

          src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af

          src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53

          src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb

          src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f

          src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0

          src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b

          src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6

          src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e

          src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e

          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 9dd71a3

          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9

          src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327

          src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8

          src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082

          Diff: https://reviews.apache.org/r/647/diff

          Testing

          -------

          Most tests pass. I have yet to run it on a cluster. Doing that now.

          Thanks,

          Michael

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/#review568 ----------------------------------------------------------- Ship it! woohoo! glad HMsg is dead! src/main/java/org/apache/hadoop/hbase/ClusterStatus.java < https://reviews.apache.org/r/647/#comment1206 > Can just use this.liveServers.values() for here and below? src/main/java/org/apache/hadoop/hbase/HServerAddress.java < https://reviews.apache.org/r/647/#comment1207 > Where is this actually used now? Should point it out here so it's clear and so that when it goes away we know we can get rid of this. src/main/java/org/apache/hadoop/hbase/HServerInfo.java < https://reviews.apache.org/r/647/#comment1208 > i see webuiport below, does this TODO still apply? src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java < https://reviews.apache.org/r/647/#comment1209 > why String and not ServerName? because master has no startcode? (i see use of ServerName for master above tho) src/main/java/org/apache/hadoop/hbase/ServerName.java < https://reviews.apache.org/r/647/#comment1210 > awesome that this is tucked away in here now src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java < https://reviews.apache.org/r/647/#comment1211 > this is because HSA actually makes a connection or does the lookup? Jonathan On 2011-04-26 23:50:23, Michael Stack wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/ ----------------------------------------------------------- (Updated 2011-04-26 23:50:23) Review request for hbase. Summary ------- This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie. RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed. We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree. Removed HMsg. HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation). Otherwise, we don't use it internally anymore. HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now. Fixed up regionserver and master startup so they now look the same. New tests Cruft cleanup. This addresses bug hbase-1502. https://issues.apache.org/jira/browse/hbase-1502 Diffs ----- src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639 src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00 src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8 src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472 src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94 src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053 src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742 src/main/java/org/apache/hadoop/hbase/Server.java df396fa src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179 src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031 src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5 src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56 src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4 src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342 src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6 src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463 src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4 src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3 src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 2273e55 src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345 src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30 src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4 src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9 src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java dada818 src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7 src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2 src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11 src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java fe807f3 src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6 src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992 src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42 src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14 src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28 src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27 src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484 src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377 src/main/resources/hbase-webapps/master/master.jsp eeabc0a src/main/resources/hbase-webapps/master/table.jsp 6fcf16f src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205 src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959 src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03 src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1 src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53 src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0 src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6 src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 9dd71a3 src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9 src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327 src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082 Diff: https://reviews.apache.org/r/647/diff Testing ------- Most tests pass. I have yet to run it on a cluster. Doing that now. Thanks, Michael
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/647/
          -----------------------------------------------------------

          (Updated 2011-04-26 23:50:23.656187)

          Review request for hbase.

          Changes
          -------

          All tests pass now. I'd like to get this patch in soon. I'm currently spending a good bit of my time trying to keep this patch up with current TRUNK. I'd rather commit and then address issues after.

          This version of the patch does one make significant change though in that it deprecates prewarmRegionCache. IMO this is a burdensome feature that is little used; i'd like to have it die off.

          Summary
          -------

          This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to
          send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie.

          RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed.
          We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree.

          Removed HMsg.

          HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation).
          Otherwise, we don't use it internally anymore.

          HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now.

          Fixed up regionserver and master startup so they now look the same.

          New tests

          Cruft cleanup.

          This addresses bug hbase-1502.
          https://issues.apache.org/jira/browse/hbase-1502

          Diffs (updated)


          src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef
          src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639
          src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00
          src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8
          src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472
          src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94
          src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053
          src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab
          src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742
          src/main/java/org/apache/hadoop/hbase/Server.java df396fa
          src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67
          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179
          src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031
          src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5
          src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5
          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a
          src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725
          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a
          src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56
          src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024
          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4
          src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f
          src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342
          src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6
          src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463
          src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4
          src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3
          src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5
          src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 2273e55
          src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345
          src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30
          src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4
          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba
          src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc
          src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e
          src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162
          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9
          src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java dada818
          src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a
          src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7
          src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2
          src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11
          src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e
          src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java fe807f3
          src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6
          src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992
          src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb
          src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d
          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42
          src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14
          src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28
          src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27
          src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484
          src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377
          src/main/resources/hbase-webapps/master/master.jsp eeabc0a
          src/main/resources/hbase-webapps/master/table.jsp 6fcf16f
          src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205
          src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe
          src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc
          src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959
          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3
          src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e
          src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa
          src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03
          src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941
          src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8
          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6
          src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d
          src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1
          src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af
          src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53
          src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb
          src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f
          src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0
          src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b
          src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6
          src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e
          src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e
          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 9dd71a3
          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9
          src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327
          src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8
          src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082

          Diff: https://reviews.apache.org/r/647/diff

          Testing
          -------

          Most tests pass. I have yet to run it on a cluster. Doing that now.

          Thanks,

          Michael

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/ ----------------------------------------------------------- (Updated 2011-04-26 23:50:23.656187) Review request for hbase. Changes ------- All tests pass now. I'd like to get this patch in soon. I'm currently spending a good bit of my time trying to keep this patch up with current TRUNK. I'd rather commit and then address issues after. This version of the patch does one make significant change though in that it deprecates prewarmRegionCache. IMO this is a burdensome feature that is little used; i'd like to have it die off. Summary ------- This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie. RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed. We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree. Removed HMsg. HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation). Otherwise, we don't use it internally anymore. HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now. Fixed up regionserver and master startup so they now look the same. New tests Cruft cleanup. This addresses bug hbase-1502. https://issues.apache.org/jira/browse/hbase-1502 Diffs (updated) src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639 src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00 src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8 src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472 src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94 src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053 src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742 src/main/java/org/apache/hadoop/hbase/Server.java df396fa src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179 src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031 src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5 src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56 src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4 src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342 src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6 src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463 src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4 src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3 src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 2273e55 src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345 src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30 src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4 src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9 src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java dada818 src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7 src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2 src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11 src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java fe807f3 src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6 src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992 src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42 src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14 src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28 src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27 src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484 src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377 src/main/resources/hbase-webapps/master/master.jsp eeabc0a src/main/resources/hbase-webapps/master/table.jsp 6fcf16f src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205 src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959 src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03 src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1 src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53 src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0 src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6 src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitLogWorker.java 9dd71a3 src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9 src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327 src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082 Diff: https://reviews.apache.org/r/647/diff Testing ------- Most tests pass. I have yet to run it on a cluster. Doing that now. Thanks, Michael
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/647/
          -----------------------------------------------------------

          (Updated 2011-04-24 23:44:37.199842)

          Review request for hbase.

          Changes
          -------

          Brought this up to the tip of TRUNK. Most tests pass. Looking into a distributed split failure.

          Summary
          -------

          This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to
          send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie.

          RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed.
          We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree.

          Removed HMsg.

          HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation).
          Otherwise, we don't use it internally anymore.

          HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now.

          Fixed up regionserver and master startup so they now look the same.

          New tests

          Cruft cleanup.

          This addresses bug hbase-1502.
          https://issues.apache.org/jira/browse/hbase-1502

          Diffs (updated)


          src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef
          src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639
          src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00
          src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8
          src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472
          src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94
          src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053
          src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab
          src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742
          src/main/java/org/apache/hadoop/hbase/Server.java df396fa
          src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67
          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179
          src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031
          src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5
          src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5
          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a
          src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725
          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a
          src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56
          src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024
          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4
          src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f
          src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342
          src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6
          src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463
          src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4
          src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3
          src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5
          src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345
          src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30
          src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4
          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba
          src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc
          src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e
          src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162
          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9
          src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a
          src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7
          src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2
          src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11
          src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e
          src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6
          src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992
          src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb
          src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d
          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42
          src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14
          src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28
          src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27
          src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484
          src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377
          src/main/resources/hbase-webapps/master/master.jsp eeabc0a
          src/main/resources/hbase-webapps/master/table.jsp 6fcf16f
          src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205
          src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe
          src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc
          src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959
          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3
          src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e
          src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa
          src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03
          src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941
          src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8
          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6
          src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d
          src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1
          src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af
          src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53
          src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb
          src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f
          src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0
          src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b
          src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6
          src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e
          src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e
          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9
          src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327
          src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8
          src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082

          Diff: https://reviews.apache.org/r/647/diff

          Testing
          -------

          Most tests pass. I have yet to run it on a cluster. Doing that now.

          Thanks,

          Michael

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/ ----------------------------------------------------------- (Updated 2011-04-24 23:44:37.199842) Review request for hbase. Changes ------- Brought this up to the tip of TRUNK. Most tests pass. Looking into a distributed split failure. Summary ------- This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie. RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed. We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree. Removed HMsg. HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation). Otherwise, we don't use it internally anymore. HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now. Fixed up regionserver and master startup so they now look the same. New tests Cruft cleanup. This addresses bug hbase-1502. https://issues.apache.org/jira/browse/hbase-1502 Diffs (updated) src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639 src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00 src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8 src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472 src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94 src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053 src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742 src/main/java/org/apache/hadoop/hbase/Server.java df396fa src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179 src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031 src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5 src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56 src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4 src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342 src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6 src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463 src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4 src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3 src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5 src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345 src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30 src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4 src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9 src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7 src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2 src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11 src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6 src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992 src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42 src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14 src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28 src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27 src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484 src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377 src/main/resources/hbase-webapps/master/master.jsp eeabc0a src/main/resources/hbase-webapps/master/table.jsp 6fcf16f src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205 src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959 src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03 src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/master/TestDeadServer.java 61baf7d src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1 src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53 src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0 src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6 src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9 src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327 src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082 Diff: https://reviews.apache.org/r/647/diff Testing ------- Most tests pass. I have yet to run it on a cluster. Doing that now. Thanks, Michael
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/647/
          -----------------------------------------------------------

          (Updated 2011-04-23 21:51:08.451039)

          Review request for hbase.

          Changes
          -------

          This diff goes against tip of TRUNK

          Summary
          -------

          This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to
          send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie.

          RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed.
          We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree.

          Removed HMsg.

          HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation).
          Otherwise, we don't use it internally anymore.

          HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now.

          Fixed up regionserver and master startup so they now look the same.

          New tests

          Cruft cleanup.

          This addresses bug hbase-1502.
          https://issues.apache.org/jira/browse/hbase-1502

          Diffs (updated)


          src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef
          src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639
          src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00
          src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8
          src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472
          src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94
          src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053
          src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab
          src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742
          src/main/java/org/apache/hadoop/hbase/Server.java df396fa
          src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67
          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179
          src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031
          src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5
          src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5
          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a
          src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725
          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a
          src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56
          src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024
          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4
          src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f
          src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342
          src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6
          src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463
          src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4
          src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3
          src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5
          src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345
          src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30
          src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4
          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba
          src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc
          src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e
          src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162
          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9
          src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a
          src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7
          src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2
          src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11
          src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e
          src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6
          src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992
          src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb
          src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d
          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42
          src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14
          src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28
          src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27
          src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484
          src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377
          src/main/resources/hbase-webapps/master/master.jsp eeabc0a
          src/main/resources/hbase-webapps/master/table.jsp 6fcf16f
          src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205
          src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe
          src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc
          src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959
          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3
          src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e
          src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa
          src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03
          src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941
          src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8
          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6
          src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1
          src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af
          src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53
          src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb
          src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f
          src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0
          src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b
          src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6
          src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e
          src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e
          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9
          src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327
          src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8
          src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082

          Diff: https://reviews.apache.org/r/647/diff

          Testing
          -------

          Most tests pass. I have yet to run it on a cluster. Doing that now.

          Thanks,

          Michael

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/ ----------------------------------------------------------- (Updated 2011-04-23 21:51:08.451039) Review request for hbase. Changes ------- This diff goes against tip of TRUNK Summary ------- This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie. RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed. We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree. Removed HMsg. HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation). Otherwise, we don't use it internally anymore. HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now. Fixed up regionserver and master startup so they now look the same. New tests Cruft cleanup. This addresses bug hbase-1502. https://issues.apache.org/jira/browse/hbase-1502 Diffs (updated) src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef src/main/java/org/apache/hadoop/hbase/HConstants.java 5701639 src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00 src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8 src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472 src/main/java/org/apache/hadoop/hbase/HServerInfo.java 0b5bd94 src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053 src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742 src/main/java/org/apache/hadoop/hbase/Server.java df396fa src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179 src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031 src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5 src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java afb666a src/main/java/org/apache/hadoop/hbase/client/HConnection.java 2bb4725 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java c348f7a src/main/java/org/apache/hadoop/hbase/client/HTable.java edacf56 src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 8df6aa4 src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java d64817f src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java c22e342 src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6 src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463 src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4 src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3 src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5 src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345 src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 58fdb30 src/main/java/org/apache/hadoop/hbase/master/DeadServer.java 05600c4 src/main/java/org/apache/hadoop/hbase/master/HMaster.java 79a48ba src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java 6c92cbc src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java 4bb072e src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 55e0162 src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 04befe9 src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7 src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2 src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java c065702 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java d0a1e11 src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 6282b9e src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6 src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992 src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42 src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14 src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 5da5e34 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28 src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27 src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484 src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java 7f5b377 src/main/resources/hbase-webapps/master/master.jsp eeabc0a src/main/resources/hbase-webapps/master/table.jsp 6fcf16f src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205 src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959 src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java 0829c8e src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03 src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 57d85d8 src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 6f718f1 src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java 6b2dc53 src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0 src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6 src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 7fc44e9 src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java bc0a327 src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082 Diff: https://reviews.apache.org/r/647/diff Testing ------- Most tests pass. I have yet to run it on a cluster. Doing that now. Thanks, Michael
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/647/
          -----------------------------------------------------------

          Review request for hbase.

          Summary
          -------

          This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to
          send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie.

          RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed.
          We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree.

          Removed HMsg.

          HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation).
          Otherwise, we don't use it internally anymore.

          HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now.

          Fixed up regionserver and master startup so they now look the same.

          New tests

          Cruft cleanup.

          This addresses bug hbase-1502.
          https://issues.apache.org/jira/browse/hbase-1502

          Diffs


          src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef
          src/main/java/org/apache/hadoop/hbase/HConstants.java ce0ea12
          src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00
          src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8
          src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472
          src/main/java/org/apache/hadoop/hbase/HServerInfo.java 7aa8bd9
          src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053
          src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab
          src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742
          src/main/java/org/apache/hadoop/hbase/Server.java df396fa
          src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67
          src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179
          src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031
          src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5
          src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5
          src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java e022fe4
          src/main/java/org/apache/hadoop/hbase/client/HConnection.java d8a2fc3
          src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 53decd6
          src/main/java/org/apache/hadoop/hbase/client/HTable.java bb3a8fa
          src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024
          src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 9576c48
          src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java db0870b
          src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java de13e27
          src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6
          src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463
          src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4
          src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3
          src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5
          src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345
          src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java e9b2af2
          src/main/java/org/apache/hadoop/hbase/master/DeadServer.java efcbb99
          src/main/java/org/apache/hadoop/hbase/master/HMaster.java 62789d3
          src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java fe5f7e1
          src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java c0e9417
          src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 7b5181a
          src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 4b8a59a
          src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a
          src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7
          src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2
          src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 516d06d
          src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 85b4144
          src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 4051e0b
          src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6
          src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992
          src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb
          src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d
          src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42
          src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14
          src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 55423af
          src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28
          src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27
          src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484
          src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6
          src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java c01fe67
          src/main/resources/hbase-webapps/master/master.jsp 198a188
          src/main/resources/hbase-webapps/master/table.jsp bba9e5b
          src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205
          src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe
          src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc
          src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e
          src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959
          src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3
          src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java a15d53a
          src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa
          src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03
          src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941
          src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 3f332ff
          src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6
          src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af
          src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java e5da229
          src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb
          src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f
          src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0
          src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b
          src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6
          src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e
          src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e
          src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 42ce89d
          src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java c2437bd
          src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8
          src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082

          Diff: https://reviews.apache.org/r/647/diff

          Testing
          -------

          Most tests pass. I have yet to run it on a cluster. Doing that now.

          Thanks,

          Michael

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/647/ ----------------------------------------------------------- Review request for hbase. Summary ------- This patch does not completely remove heartbeats. It unburdens the heartbeat of control messages; now heartbeat is used to send the master load only (At most recent hackathon we had rough agreement that we'd keep heartbeat to carry load)... if we miss some, no biggie. RPC version changed on HMasterRegionInfo since the regionServerStartup and regionServerReport arguments have changed. We pass a String now instead of HServerAddress so this should help with our DNS issues where the two sides disagree. Removed HMsg. HServerAddress as been sort_of_deprecated. Its in our API so can't remove it easily (its embedded inside HRegionLocation). Otherwise, we don't use it internally anymore. HServerInfo is deprecated. Server meta data is now available in new class ServerName and load lives apart from HSI now. Fixed up regionserver and master startup so they now look the same. New tests Cruft cleanup. This addresses bug hbase-1502. https://issues.apache.org/jira/browse/hbase-1502 Diffs src/main/java/org/apache/hadoop/hbase/ClusterStatus.java 26a8bef src/main/java/org/apache/hadoop/hbase/HConstants.java ce0ea12 src/main/java/org/apache/hadoop/hbase/HMsg.java 87beb00 src/main/java/org/apache/hadoop/hbase/HRegionLocation.java bd353b8 src/main/java/org/apache/hadoop/hbase/HServerAddress.java 7f8a472 src/main/java/org/apache/hadoop/hbase/HServerInfo.java 7aa8bd9 src/main/java/org/apache/hadoop/hbase/HServerLoad.java 2372053 src/main/java/org/apache/hadoop/hbase/LocalHBaseCluster.java 0d696ab src/main/java/org/apache/hadoop/hbase/MasterAddressTracker.java 1da9742 src/main/java/org/apache/hadoop/hbase/Server.java df396fa src/main/java/org/apache/hadoop/hbase/ServerName.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/avro/AvroUtil.java d7a1e67 src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java be31179 src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java c2ee031 src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 6e22cf5 src/main/java/org/apache/hadoop/hbase/catalog/RootLocationEditor.java aee64c5 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java e022fe4 src/main/java/org/apache/hadoop/hbase/client/HConnection.java d8a2fc3 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 53decd6 src/main/java/org/apache/hadoop/hbase/client/HTable.java bb3a8fa src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java 6c62024 src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserver.java 9576c48 src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.java db0870b src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java de13e27 src/main/java/org/apache/hadoop/hbase/executor/RegionTransitionData.java a55f9d6 src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java d8f8463 src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java ec28de4 src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java 25139b3 src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 663cab5 src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java 66a3345 src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java e9b2af2 src/main/java/org/apache/hadoop/hbase/master/DeadServer.java efcbb99 src/main/java/org/apache/hadoop/hbase/master/HMaster.java 62789d3 src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java fe5f7e1 src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.java c0e9417 src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 7b5181a src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 4b8a59a src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java eb01a6a src/main/java/org/apache/hadoop/hbase/master/handler/OpenedRegionHandler.java c478ab7 src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java 60662c2 src/main/java/org/apache/hadoop/hbase/master/handler/SplitRegionHandler.java fee0b3d src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 516d06d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 85b4144 src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java 4051e0b src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java 1b521f6 src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 7569992 src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java 548c8eb src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java f282c6d src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 78c3b42 src/main/java/org/apache/hadoop/hbase/rest/StorageClusterStatusResource.java 578ee14 src/main/java/org/apache/hadoop/hbase/util/Addressing.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 55423af src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java b624d28 src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java baf0c27 src/main/java/org/apache/hadoop/hbase/zookeeper/RegionServerTracker.java 0437484 src/main/java/org/apache/hadoop/hbase/zookeeper/RootRegionTracker.java 692b608 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKAssign.java 34e17b6 src/main/java/org/apache/hadoop/hbase/zookeeper/ZKUtil.java c01fe67 src/main/resources/hbase-webapps/master/master.jsp 198a188 src/main/resources/hbase-webapps/master/table.jsp bba9e5b src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 28d2205 src/test/java/org/apache/hadoop/hbase/TestHRegionLocation.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerAddress.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHServerInfo.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java 11b94fe src/test/java/org/apache/hadoop/hbase/TestSerialization.java 40874fc src/test/java/org/apache/hadoop/hbase/TestServerName.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java e25184e src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTrackerOnCluster.java 66c6959 src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditor.java 60320a3 src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserver.java a15d53a src/test/java/org/apache/hadoop/hbase/master/BROKE_FIX_TestKillingServersFromMaster.java 21b76fa src/test/java/org/apache/hadoop/hbase/master/OOMEHMaster.java bf5ed03 src/test/java/org/apache/hadoop/hbase/master/TestActiveMasterManager.java 1a19941 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 3f332ff src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java 915cdf6 src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java a1bb6af src/test/java/org/apache/hadoop/hbase/master/TestLoadBalancer.java e5da229 src/test/java/org/apache/hadoop/hbase/master/TestLogsCleaner.java 19220fb src/test/java/org/apache/hadoop/hbase/master/TestMaster.java c4ea83f src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 5a334e0 src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java dff6c1b src/test/java/org/apache/hadoop/hbase/master/TestRollingRestart.java 6089ae6 src/test/java/org/apache/hadoop/hbase/regionserver/TestMasterAddressManager.java 319a74e src/test/java/org/apache/hadoop/hbase/regionserver/TestScanner.java 7ff6a2e src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java 42ce89d src/test/java/org/apache/hadoop/hbase/regionserver/handler/TestOpenRegionHandler.java c2437bd src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 20a1ff8 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java a055082 Diff: https://reviews.apache.org/r/647/diff Testing ------- Most tests pass. I have yet to run it on a cluster. Doing that now. Thanks, Michael
          Hide
          stack added a comment -

          Marking patch available.

          Show
          stack added a comment - Marking patch available.
          Hide
          stack added a comment -

          Added a patch that is ready for review to https://reviews.apache.org/r/647/ I'm still testing but its ready for input.

          Show
          stack added a comment - Added a patch that is ready for review to https://reviews.apache.org/r/647/ I'm still testing but its ready for input.
          Hide
          stack added a comment -

          Compiles. Not ready for review yet.

          Show
          stack added a comment - Compiles. Not ready for review yet.
          Hide
          stack added a comment -

          The Master needs to know cluster load for load balancing and so we can display load aggregate in the UI. Currently this is carried on the back of the heatbeat as a serialized HServerLoad. The intent was to put this loading elsewhere than up on the heartbeat. Thought was to put it out in zk.

          At the hackathon, it was thought that on a cluster of any reasonable size, the number of writes would be too high (Think 700 node cluster writing load every 1-5 seconds). Todd raised the issue after talking with Patrick; zk blocks reads while taking on writes so our read rate would go down.

          Chatting more, the thought was to just leave up the heartbeat connection – call it something else though (jgray) – but remove HMsg and just pass HSLs. We talked of master querying on a period all RSs but for now, we have something that works so just keep it up it was thought.

          Show
          stack added a comment - The Master needs to know cluster load for load balancing and so we can display load aggregate in the UI. Currently this is carried on the back of the heatbeat as a serialized HServerLoad. The intent was to put this loading elsewhere than up on the heartbeat. Thought was to put it out in zk. At the hackathon, it was thought that on a cluster of any reasonable size, the number of writes would be too high (Think 700 node cluster writing load every 1-5 seconds). Todd raised the issue after talking with Patrick; zk blocks reads while taking on writes so our read rate would go down. Chatting more, the thought was to just leave up the heartbeat connection – call it something else though (jgray) – but remove HMsg and just pass HSLs. We talked of master querying on a period all RSs but for now, we have something that works so just keep it up it was thought.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #1767 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1767/)

          Show
          Hudson added a comment - Integrated in HBase-TRUNK #1767 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1767/ )
          Hide
          stack added a comment -

          Here is latest. Putting aside for the moment. Going to work on splits, load, and shutdown in separate issues rather than bundle them in as part of this patch.

          Show
          stack added a comment - Here is latest. Putting aside for the moment. Going to work on splits, load, and shutdown in separate issues rather than bundle them in as part of this patch.
          Hide
          stack added a comment -

          Compiles. Some tests pass. Working on the rest.

          Show
          stack added a comment - Compiles. Some tests pass. Working on the rest.
          Hide
          stack added a comment -

          Patch does not compile yet. Still have shutdown and split to do. Here is what I have
          so far.

          Removing heartbeats has brought on a refactoring. A couple of our fundamental types
          make little sense in absence of heartbeating. We've also been abusing core types such
          as HServerAddress serializing it up into zookeeper, a problematic behavior given deserializing
          will bring on a resolve.

          An HServerAddress is a Writable InetSocketAddress. We should just be using ISA. Passing
          an HSA that will resolve when deserialized on other side is something we need to move
          away from. Lets just use ISA from here on out. HSA was not removed because its in public
          API in a few places; it shouldn't be.

          HServerInfo has deprecated and removed from all locations but where its part of public API.
          It hosts an HServerAddress, HServerLoad, and webuiport. It was mainly passed by the regionserver
          heartbeating. Now we don't heartbeat, passing load in HServerInfo doesn't make sense. Also
          the webuiport needs to be obtained elsewhere than on heartbeat (its up in zk in the rs znode data).
          Finally, the passing of an HServerAddress inside the HServerInfo is problematic (See above).

          HServerLoad has been removed. It was passed inside HServerInfo on each heartbeat.
          TODO: replace with alternative. I thought of writing 'load' to zk every minute or so but
          that could get hefty if lots of servers. Still might go this route. Wanted to do it w/
          JSON but HSL wasn't easy to serialize with its aggregating and internal RegionLoad.

          Patch includes new class used identifying servers. Its a formalization of our special
          'servername' String. The class is called ServerName. Its not a Writable. This class is
          used near everywhere in place of HSA and/or HSI identifying servers. A ServerName can be
          made from the server + startcode columns in .META. Its the name of the RS znode in zookeeper.
          A bunch of messing with hostname and port only has been removed.

          Changes CP Interfaces to use ServerName instead of HSI.

          Changes the HMasterRegionInterface to remove the heartbeat. Upped this interfaces
          version number.

          Removed from ZKUtil all the getAddress methods.

          Redid the master join cluster code. Previous we relied on HServerInfo on server
          registration but this is not available any more. Instead scrape .META. and RIT
          to figure whether failover or fresh cluster start.

          Changd the way we do our address in Master and RS. Removed HSI and HSA. Using
          InetSocketAddress instead. Cleaned a bunch of crud from RS in particular.
          Made this code same in master and rs.

          RegionServer now registers in ZK only after registering first with master.
          It will use the hostname that the master sees the regionserver in its znode
          name. RS is careful to use the hostname the master passed us writing .META.
          (doesn't use the RS hostname – there is no RS hostname kept any more other than
          what is in the RS ISA).

          RegionServerTracker now keeps running list of servers. Also has znode content
          which currently is the webuiport only.

          Show
          stack added a comment - Patch does not compile yet. Still have shutdown and split to do. Here is what I have so far. Removing heartbeats has brought on a refactoring. A couple of our fundamental types make little sense in absence of heartbeating. We've also been abusing core types such as HServerAddress serializing it up into zookeeper, a problematic behavior given deserializing will bring on a resolve. An HServerAddress is a Writable InetSocketAddress. We should just be using ISA. Passing an HSA that will resolve when deserialized on other side is something we need to move away from. Lets just use ISA from here on out. HSA was not removed because its in public API in a few places; it shouldn't be. HServerInfo has deprecated and removed from all locations but where its part of public API. It hosts an HServerAddress, HServerLoad, and webuiport. It was mainly passed by the regionserver heartbeating. Now we don't heartbeat, passing load in HServerInfo doesn't make sense. Also the webuiport needs to be obtained elsewhere than on heartbeat (its up in zk in the rs znode data). Finally, the passing of an HServerAddress inside the HServerInfo is problematic (See above). HServerLoad has been removed. It was passed inside HServerInfo on each heartbeat. TODO: replace with alternative. I thought of writing 'load' to zk every minute or so but that could get hefty if lots of servers. Still might go this route. Wanted to do it w/ JSON but HSL wasn't easy to serialize with its aggregating and internal RegionLoad. Patch includes new class used identifying servers. Its a formalization of our special 'servername' String. The class is called ServerName. Its not a Writable. This class is used near everywhere in place of HSA and/or HSI identifying servers. A ServerName can be made from the server + startcode columns in .META. Its the name of the RS znode in zookeeper. A bunch of messing with hostname and port only has been removed. Changes CP Interfaces to use ServerName instead of HSI. Changes the HMasterRegionInterface to remove the heartbeat. Upped this interfaces version number. Removed from ZKUtil all the getAddress methods. Redid the master join cluster code. Previous we relied on HServerInfo on server registration but this is not available any more. Instead scrape .META. and RIT to figure whether failover or fresh cluster start. Changd the way we do our address in Master and RS. Removed HSI and HSA. Using InetSocketAddress instead. Cleaned a bunch of crud from RS in particular. Made this code same in master and rs. RegionServer now registers in ZK only after registering first with master. It will use the hostname that the master sees the regionserver in its znode name. RS is careful to use the hostname the master passed us writing .META. (doesn't use the RS hostname – there is no RS hostname kept any more other than what is in the RS ISA). RegionServerTracker now keeps running list of servers. Also has znode content which currently is the webuiport only.
          Hide
          Jonathan Gray added a comment -

          Got it. That sounds like a good start and removing the heartbeat/HMsg is far more critical than the startup RPC. I remember some issue in the past where it was odd that discovery was RPC but failure was ZK... but don't recall what exactly it was now.

          +1 on ZK content being JSON serialized.

          Just to bring it up since it's loosely related to this stuff, I'm of the opinion that a second monitor process is eventually going to be necessary. ZK timeouts are just too high and there are many cases where if we could take GC pauses out of the equation, we could have much faster failure detection. A second process that would not have any GC issues could have an ephemeral node with a much smaller timeout, or it could monitor the RS process and system itself. I don't see another way towards reducing failure detection times without triggering false-positives when the RS is in a GC pause (a "recoverable" fault).

          On an even more unrelated note, we could have some kind of metric (or this other process could figure out) how often GC pauses are happening / for how long (either through a looping sleep() thread or an RPC to the process) and use that as an additional balancing metric. Or we could have it so once it passes a threshold, we shed the regions off of that RS (actually flushing instead of needing replay), and then restart the RS process.

          Show
          Jonathan Gray added a comment - Got it. That sounds like a good start and removing the heartbeat/HMsg is far more critical than the startup RPC. I remember some issue in the past where it was odd that discovery was RPC but failure was ZK... but don't recall what exactly it was now. +1 on ZK content being JSON serialized. Just to bring it up since it's loosely related to this stuff, I'm of the opinion that a second monitor process is eventually going to be necessary. ZK timeouts are just too high and there are many cases where if we could take GC pauses out of the equation, we could have much faster failure detection. A second process that would not have any GC issues could have an ephemeral node with a much smaller timeout, or it could monitor the RS process and system itself. I don't see another way towards reducing failure detection times without triggering false-positives when the RS is in a GC pause (a "recoverable" fault). On an even more unrelated note, we could have some kind of metric (or this other process could figure out) how often GC pauses are happening / for how long (either through a looping sleep() thread or an RPC to the process) and use that as an additional balancing metric. Or we could have it so once it passes a threshold, we shed the regions off of that RS (actually flushing instead of needing replay), and then restart the RS process.
          Hide
          stack added a comment -

          Yeah, 'registration' is via an RPC. The RS sends over startcode and port only (should probably pass the webui port too). The master returns the hostname only. On receipt, the RS writes up into ZK, a znode whose name is the 'servername – i.e. the master passed domain, port, and startcode – and whose content is the RS 'load' updated every 5 minutes or so. I think load should be JSON serialized (I think all zk content should be JSON serialized)

          I'm willing to entertain suggestions on how we might do away with the startup RPC but we need to get to the RS the hostname the Master sees; currently it does this by doing lookup on the remote ip the RPCServer sees on registration RPC. The RS needs what the master has for a hostname so when it writes .META., it writes the same name the master will write.

          Show
          stack added a comment - Yeah, 'registration' is via an RPC. The RS sends over startcode and port only (should probably pass the webui port too). The master returns the hostname only. On receipt, the RS writes up into ZK, a znode whose name is the 'servername – i.e. the master passed domain, port, and startcode – and whose content is the RS 'load' updated every 5 minutes or so. I think load should be JSON serialized (I think all zk content should be JSON serialized) I'm willing to entertain suggestions on how we might do away with the startup RPC but we need to get to the RS the hostname the Master sees; currently it does this by doing lookup on the remote ip the RPCServer sees on registration RPC. The RS needs what the master has for a hostname so when it writes .META., it writes the same name the master will write.
          Hide
          Jonathan Gray added a comment -

          (yay HMsg is gone!)

          Show
          Jonathan Gray added a comment - (yay HMsg is gone!)
          Hide
          Jonathan Gray added a comment -

          Only did cursory look at patch, but looks like this still has it so startup is via RPC but shutdown (will eventually be) via ZK... What exactly will be transmitted each way in that startup RPC? Is it necessary?

          Also, how does master get server load info now?

          Show
          Jonathan Gray added a comment - Only did cursory look at patch, but looks like this still has it so startup is via RPC but shutdown (will eventually be) via ZK... What exactly will be transmitted each way in that startup RPC? Is it necessary? Also, how does master get server load info now?
          Hide
          stack added a comment -

          Fixup of HServerInfo, HServerAddress, HRegionLoad, and HRegionLocation classes – tightened cohesion, added javadoc. Added tests. More purging of code. Still have split and shutdown to do. Was going to persist to zk using JSON instead of Writables.

          Show
          stack added a comment - Fixup of HServerInfo, HServerAddress, HRegionLoad, and HRegionLocation classes – tightened cohesion, added javadoc. Added tests. More purging of code. Still have split and shutdown to do. Was going to persist to zk using JSON instead of Writables.
          Hide
          stack added a comment -

          Made this a blocker. Need this to fix up the bug where we'd see splits before region had successfully opened.

          Show
          stack added a comment - Made this a blocker. Need this to fix up the bug where we'd see splits before region had successfully opened.
          Hide
          stack added a comment -

          Here is a start – the fun part. Removed HMsg and then redid the HMasterRegionInterface to remove the heartbeating. Was able to remove a bunch of other crud from Master and Regionserver. TODO: Splits, shutdown, and load. The report into the master by RS remains. It'll be passed back the hostname to use. It'll add this up into its znode (it'll only add znode after it has registered w/ Master). The znode will then be updated by the RS on a period with its load info. Master loadbalancer will read it from there. Regards Splits, after talking w/ Jon, its a new RIT state. Shutdown, I'll have think about it and write something up.

          Show
          stack added a comment - Here is a start – the fun part. Removed HMsg and then redid the HMasterRegionInterface to remove the heartbeating. Was able to remove a bunch of other crud from Master and Regionserver. TODO: Splits, shutdown, and load. The report into the master by RS remains. It'll be passed back the hostname to use. It'll add this up into its znode (it'll only add znode after it has registered w/ Master). The znode will then be updated by the RS on a period with its load info. Master loadbalancer will read it from there. Regards Splits, after talking w/ Jon, its a new RIT state. Shutdown, I'll have think about it and write something up.
          Hide
          stack added a comment -

          Upping priority and changed from wish to task. See HBASE-3368 for a good illustration of why getting rid of heartbeats is a necessary task rather than a wish.

          Show
          stack added a comment - Upping priority and changed from wish to task. See HBASE-3368 for a good illustration of why getting rid of heartbeats is a necessary task rather than a wish.
          Hide
          Jonathan Gray added a comment -

          We're also using the regionServerStartup or regionServerReport to determine an RS is online (we use the ZK node to determine when it dies). This stuff would also need some cleanup/rework to completely drop heartbeats.

          Show
          Jonathan Gray added a comment - We're also using the regionServerStartup or regionServerReport to determine an RS is online (we use the ZK node to determine when it dies). This stuff would also need some cleanup/rework to completely drop heartbeats.
          Hide
          stack added a comment -

          We are almost there. If we could get rid of the split message and shutdown being passed by HMsg, we'd be able to drop HMsg and just rely completely on zk client ping.

          Show
          stack added a comment - We are almost there. If we could get rid of the split message and shutdown being passed by HMsg, we'd be able to drop HMsg and just rely completely on zk client ping.
          Hide
          stack added a comment -

          Post hbase-2692, heartbeats:

          + Carry to the master the HServerInfo from each regionserver.
          + When a split, a message from rs tells master about the it; master uses info to update its inmemory map of cluster
          + Master tells rs to shutdown (needed currently if rs is carrying a meta region)

          I was thinking that RSs could publish load info to zk on a period – every minute or 5 (some fraction of the balancer interval).

          For split info, need to flag master somehow and needs to be mechanism where the split can be rediscovered if missed for some reason (The anti-entropy doohickey where clients tell master of 'problems' would be one such way).

          On shutdown message, I need to do a little more smarts around shutdown and that should eliminate the shutdown message (need to expose shutdown as something a client can invoke against an RS).

          Show
          stack added a comment - Post hbase-2692, heartbeats: + Carry to the master the HServerInfo from each regionserver. + When a split, a message from rs tells master about the it; master uses info to update its inmemory map of cluster + Master tells rs to shutdown (needed currently if rs is carrying a meta region) I was thinking that RSs could publish load info to zk on a period – every minute or 5 (some fraction of the balancer interval). For split info, need to flag master somehow and needs to be mechanism where the split can be rediscovered if missed for some reason (The anti-entropy doohickey where clients tell master of 'problems' would be one such way). On shutdown message, I need to do a little more smarts around shutdown and that should eliminate the shutdown message (need to expose shutdown as something a client can invoke against an RS).
          Hide
          Jonathan Gray added a comment -

          Following HBASE-2692, I think the only thing we're really using the heartbeat for is the mostLoaded servers. But we're actually tracking all assignment in-memory anyways, so maybe we can do this now? Shutdown stuff could come on it's own RPC?

          Show
          Jonathan Gray added a comment - Following HBASE-2692 , I think the only thing we're really using the heartbeat for is the mostLoaded servers. But we're actually tracking all assignment in-memory anyways, so maybe we can do this now? Shutdown stuff could come on it's own RPC?
          Hide
          stack added a comment -

          Moved from 0.21 to 0.22 just after merge of old 0.20 branch into TRUNK.

          Show
          stack added a comment - Moved from 0.21 to 0.22 just after merge of old 0.20 branch into TRUNK.
          Hide
          stack added a comment -

          Not for a point release. Lets do in 0.21.

          Show
          stack added a comment - Not for a point release. Lets do in 0.21.
          Hide
          ryan rawson added a comment -

          right now the master makes assignment decisions during the 'check in' RPC. Basically the master doesn't proactively assign regions, instead it passively waits for regionservers to call the master.

          We'd also need new RPC so a master can be much more proactive about assigning regions. This could reduce assignment time by seconds .

          Show
          ryan rawson added a comment - right now the master makes assignment decisions during the 'check in' RPC. Basically the master doesn't proactively assign regions, instead it passively waits for regionservers to call the master. We'd also need new RPC so a master can be much more proactive about assigning regions. This could reduce assignment time by seconds .

            People

            • Assignee:
              stack
              Reporter:
              Nitay Joffe
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development