[HBASE-18110] [AMv2] Reenable tests temporarily disabled

• Type: Umbrella
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0-beta-1, 2.0.0
• Component/s: Region Assignment
• Assignee: Michael Stack
• Reporter: Michael Stack
• Created: 25/May/17 00:15
• Updated: 05/Apr/18 19:37
• Resolved: 15/Dec/17 18:03

Description

We disabled tests that didn't make sense or relied on behavior not supported by AMv2. Revisit and reenable after AMv2 gets committed. Here is the set (from https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.rsj53tx4vlwj)

testAllFavoredNodesDead and testAllFavoredNodesDeadMasterRestarted and testMisplacedRegions in TestFavoredStochasticLoadBalancer … not sure what this about.
testRegionNormalizationMergeOnCluster in TestSimpleRegionNormalizerOnCluster disabled for now till we fix up Merge.
testMergeWithReplicas in TestRegionMergeTransactionOnCluster because don't know how it is supposed to work.
Admin#close does not update Master. Causes testHBaseFsckWithFewerMetaReplicaZnodes in TestMetaWithReplicas to fail (Master gets report about server closing when it didn’t run the close – gets freaked out).
Disabled/Ignore TestRSGroupsOfflineMode#testOffline; need to dig in on what offline is.
Disabled/Ignore TestRSGroups.
All tests that have to do w/ fsck:TestHBaseFsckTwoRS, TestOfflineMetaRebuildBase TestHBaseFsckReplicas, TestOfflineMetaRebuildOverlap, testChangingReplicaCount in TestMetaWithReplicas (internally it is doing fscks which are killing RS)...
FSCK test testHBaseFsckWithExcessMetaReplicas in TestMetaWithReplicas.
So is testHBaseFsckWithFewerMetaReplicas in same class.
TestHBaseFsckOneRS is fsck. Disabled.
TestOfflineMetaRebuildHole is about rebuilding hole with fsck.
Master carries meta:
TestRegionRebalancing is disabled because doesn't consider the fact that Master carries system tables only (fix of average in RegionStates brought out the issue).
Disabled testMetaAddressChange in TestMetaWithReplicas because presumes can move meta... you can't
TestAsyncTableGetMultiThreaded wants to move hbase:meta...Balancer does NPEs. AMv2 won't let you move hbase:meta off Master.
Disabled parts of...testCreateTableWithMultipleReplicas in TestMasterOperationsForRegionReplicas There is an issue w/ assigning more replicas if number of replicas is changed on us. See '/* DISABLED!!!!! FOR NOW!!!!'.
Disabled TestCorruptedRegionStoreFile. Depends on a half-implemented reopen of a region when a store file goes missing; TODO.
testRetainAssignmentOnRestart in TestRestartCluster does not work. AMv2 does retain semantic differently. Fix. TODO.
TestMasterFailover needs to be rewritten for AMv2. It uses tricks not ordained when up on AMv2. The test is also hobbled by fact that we religiously enforce that only master can carry meta, something we are lose about in old AM.
Fix Ignores in TestServerCrashProcedure. Master is different now.
Offlining is done differently now: Because of this disabled testOfflineRegion in TestAsyncRegionAdminApi
Skipping delete of table after test in TestAccessController3 because of access issues w/ AMv2. AMv1 seems to crash servers on exit too for same lack of auth perms but AMv2 gets hung up. TODO. See cleanUp method.
TestHCM#testMulti and TestHCM
Fix TestMasterMetrics. Stuff is different now around startup which messes up this test. Disabled two of three tests.
I tried to fix TestMasterBalanceThrottling but it looks like SimpleLoadBalancer is borked whether AMv2 or not.
Disabled testPickers in TestFavoredStochasticBalancerPickers. It hangs.

Issue Links

• contains: HBASE-18231 Deprecate and throw unsupported operation when Admin#closeRegion is called.
• contains: HBASE-18350 RSGroups are broken under AMv2
• contains: HBASE-18575 [AMv2] Enable and fix TestRestartCluster#testRetainAssignmentOnRestart on master
• contains: HBASE-18301 Enable TestSimpleRegionNormalizerOnCluster#testRegionNormalizationMergeOnCluster that was disabled by Proc-V2 AM in HBASE-14614
• contains: HBASE-18351 Fix tests that carry meta in Master that were disabled by Proc-V2 AM in HBASE-14614
• contains: HBASE-18352 Enable TestMasterOperationsForRegionReplicas#testCreateTableWithMultipleReplicas disabled by Proc-V2 AM in HBASE-14614
• contains: HBASE-18353 Enable TestCorruptedRegionStoreFile that were disabled by Proc-V2 AM in HBASE-14614
• contains: HBASE-18354 Fix TestMasterMetrics that were disabled by Proc-V2 AM in HBASE-14614
• contains: HBASE-18355 Enable export snapshot tests that were disabled by Proc-V2 AM in HBASE-14614
• contains: HBASE-18356 Enable TestFavoredStochasticBalancerPickers#testPickers that was disabled by Proc-V2 AM in HBASE-14614
• contains: HBASE-18357 Enable disabled tests in TestHCM that were disabled by Proc-V2 AM in HBASE-14614
• relates to: HBASE-19269 Reenable TestShellRSGroups
• relates to: HBASE-19272 Deal with HBCK tests disabled by HBASE-14614 AMv2 when HBCK works again...
• relates to: HBASE-18349 Enable disabled tests in TestFavoredStochasticLoadBalancer that were disabled by Proc-V2 AM in HBASE-14614
• relates to: HBASE-19231 Redo TestMetaWithReplicas#testChangingReplicaCount; uses HBCK to validate
• relates to: HBASE-18244 org.apache.hadoop.hbase.client.rsgroup.TestShellRSGroups hangs/fails

Sub-Tasks

• Enable Replica tests that were disabled by Proc-V2 AM in HBASE-14614: Resolved
• Reenable TestRegionMergeTransactionOnCluster#testMergeWithReplicas disable by HBASE-14614: Resolved
• Reenable cleanup in test teardown in TestAccessController3 disabled by HBASE-14614: Resolved
• Remove TestAssignmentManager#testGoodSplit, which no longer make sense: Resolved