[PHOENIX-5137] Index Rebuilder scan increases data table region split time

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.14.1
• Fix Version/s: 4.15.0, 5.1.0, 4.14.2
• Assignee: Kiran Kumar Maturi
• Reporter: Kiran Kumar Maturi
• Created: 13/Feb/19 11:42
• Updated: 30/May/19 18:40
• Resolved: 19/Apr/19 00:27

Description

lhofhansl vincentpoon tdsilva please review

In order to differentiate between the index rebuilder retries  (UngroupedAggregateRegionObserver.rebuildIndices()) and commits that happen in the loop of UngroupedAggregateRegionObserver.doPostScannerOpen() as part of  PHOENIX-4600 blockingMemstoreSize was set to -1 for rebuildIndices;

commitBatchWithRetries(region, mutations, -1);

blocks the region split as the check for region closing does not happen  blockingMemstoreSize > 0

for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() > blockingMemstoreSize && i < 30; i++) {
  try{
   checkForRegionClosing();
   ....

Plan is to have the check for region closing at least once before committing the batch

checkForRegionClosing();
for (int i = 0; blockingMemstoreSize > 0 && region.getMemstoreSize() > blockingMemstoreSize && i < 30; i++) {
  try{
   checkForRegionClosing();
   ....

Steps to reproduce
1. Create a table with one index (startime)
2. Add 1-2 million rows
3. Wait till the index is active
4. Disable the index with start time (noted in step 1)
5. Once the rebuilder starts split data table region

Repeat the steps again after applying the patch to check the difference.