[GEODE-6186] Reduce the number of EntryNotFoundExceptions during AsyncEventQueue batch processing with conflation enabled

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.9.0
• Component/s: wan
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Barrett Oglesby
• Created: 11/Dec/18 19:29
• Updated: 30/Apr/19 20:59
• Resolved: 30/Apr/19 20:59

Description

Reduce the number of EntryNotFoundExceptions during AsyncEventQueue batch processing with conflation enabled

This test:

3000 iterations of putAlls with the same 1500 keys into a partitioned region attached to async-event-queue:

<async-event-queue id="test" batch-size="100000" batch-time-interval="250" dispatcher-threads="1" parallel="true" enable-batch-conflation="true">

Produces these numbers in the current code (4 different runs):

numBatches=645; numENFEs=8622196; totalPeekTime=178517; averagePeekTime=276; totalProcessBatchTime=38936; averageProcessBatchTime=60
numBatches=660; numENFEs=8467986; totalPeekTime=182985; averagePeekTime=277; totalProcessBatchTime=34335; averageProcessBatchTime=52
numBatches=646; numENFEs=8563364; totalPeekTime=179624; averagePeekTime=278; totalProcessBatchTime=37342; averageProcessBatchTime=57
numBatches=632; numENFEs=8716942; totalPeekTime=175570; averagePeekTime=277; totalProcessBatchTime=39732; averageProcessBatchTime=62

After some changes mainly in BucketRegionQueue:

numBatches=782; numENFEs=3621039; totalPeekTime=195760; averagePeekTime=250; totalProcessBatchTime=18724; averageProcessBatchTime=23
numBatches=791; numENFEs=3604933; totalPeekTime=197980; averagePeekTime=250; totalProcessBatchTime=18587; averageProcessBatchTime=23
numBatches=790; numENFEs=3600038; totalPeekTime=197774; averagePeekTime=250; totalProcessBatchTime=18611; averageProcessBatchTime=23
numBatches=795; numENFEs=3584490; totalPeekTime=199060; averagePeekTime=250; totalProcessBatchTime=18063; averageProcessBatchTime=22

numBatches is the number of batches peeked
numENFEs is the number of EntryNotFoundExceptions thrown
totalPeekTime is the total time to peek all batches
averagePeekTime is the average time to peek a batch
totalProcessBatchTime is the total time to process all batches
averageProcessBatchTime is the average time to process a batch (includes listener callback and remove from queue)

Issue Links

• links to: GitHub Pull Request #2987