[IGNITE-17291] Implement metastorage cursor batching

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-beta1
• Labels: ignite-3
• Assignee: Denis Chudov
• Reporter: Alexander Lapin
• Reviewer: Alexander Lapin
• Created: 01/Jul/22 10:04
• Updated: 13/Oct/22 12:00
• Resolved: 11/Jul/22 16:27

Description

In order to increase metaStorage.range() performance that currently retrieves entries one by one it's possible to implement simple batching. As an initial solution we might hardcode the batch-size.

Basically speaking it's required to update CursorNextCommand.

Instead of

Entry e = (Entry) cursorDesc.cursor().next();

clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(), e.updateCounter())); 

we might use something similar to

    List<SingleEntryResponse> batch = new ArrayList<>(RANGE_CURSOR_BATCH_SIZE);

    for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
        Entry e = (Entry) cursorDesc.cursor().next();

        batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(), e.updateCounter()));

        if (! cursorDesc.cursor.hasNext()) {
            break;
        }
    }

clo.result(new MultipleEntryResponse(batch));

It's not trivial to reimplement rocks cursors to also use batching, however it's not that important because rocks cursors are always local.

 

Besides that it's required to update org.apache.ignite.internal.metastorage.client.CursorImpl with client-side-iteration over batched data and requesting new portion if nothing left.

Issue Links

• relates to: IGNITE-16760 Performance degradation of IgniteTables#tables after configuration changes
• links to: GitHub Pull Request #931
• links to: PR