[IGNITE-21521] Wrong update order in upsertAll and DataStreamer for a new key

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-beta1
• Fix Version/s: 3.0.0-beta2
• Labels: ignite-3
• Assignee: Pavel Tupitsyn
• Reporter: Pavel Tupitsyn
• Reviewer: Alexey Scherbakov
• Created: 13/Feb/24 07:34
• Updated: 05/Apr/24 07:35
• Resolved: 21/Mar/24 14:46

Description

Add the following test to ItInternalTableTest to reproduce:

    @Test
    public void upsertAllOrderTest() {
        RecordView<Tuple> view = table.recordView();
        InternalTable internalTable = ((TableViewInternal) table).internalTable();
        List<BinaryRowEx> rows = new ArrayList<>();

        int count = 100;
        int lastId = count - 1;
        long id = 12345;

        for (int i = 0; i < count; i++) {
            rows.add(createKeyValueRow(id, i, "row-" + i));
        }

        internalTable.upsertAll(rows, null).join();

        Tuple res = view.get(null, Tuple.create().set("key", id));
        assertEquals(lastId, res.intValue("valInt"));
        assertEquals("row-" + lastId, res.stringValue("valStr"));
    }

As a user, I expect to observe the last value for the given key. Instead, there is a random value.

This is caused by PartitionReplicaListener - when there is no row with specified key, we generate a new random RowId:

RowId rowId0 = insert ? new RowId(partId(), UUID.randomUUID()) : rowId

Then this row id is stored in a hash set, which results in a random update order. The same applies to the DataStreamer, which goes through InternalTableImpl.updateAll.

Issue Links

• is related to: IGNITE-21401 DataStreamer data removal: improve RW_UPSERT_ALL to support delete
• is related to: IGNITE-21403 DataStreamer data removal: update public API
• relates to: IGNITE-21992 Data Streamer removal does not work for a new key in the same batch
• links to: GitHub Pull Request #3439