[PHOENIX-1149] Index mutations should be committed as batches while building local index

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.1.0
• Assignee: rajeshbabu
• Reporter: rajeshbabu
• Created: 06/Aug/14 06:27
• Updated: 21/Nov/15 02:15
• Resolved: 06/Aug/14 16:45

Description

Currently while building local indexes batchSize is 0 so we are not committing the index mutations until full region scan is done and which may cause high memory usage or OOM with huge data.

                        // Commit in batches based on UPSERT_BATCH_SIZE_ATTRIB in config
                        if (!indexMutations.isEmpty() && batchSize > 0 && indexMutations.size() % batchSize == 0) {
                            HRegion indexRegion = getIndexRegion(c.getEnvironment());
                            // Get indexRegion corresponding to data region
                            commitBatch(indexRegion, indexMutations, null);
                            indexMutations.clear();
                        }

Thanks for finding this mujtabachohan.

Issue Links

• is related to: PHOENIX-933 Enhance Local index support