[IGNITE-21992] Data Streamer removal does not work for a new key in the same batch

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-beta1
• Fix Version/s: 3.0.0-beta2
• Component/s: streaming
• Labels: ignite-3
• Assignee: Pavel Tupitsyn
• Reporter: Pavel Tupitsyn
• Reviewer: Alexey Scherbakov
• Created: 05/Apr/24 07:35
• Updated: 08/May/24 09:45
• Resolved: 08/May/24 09:45

Description

When remove operation follows insert in the same batch, remove is ignored. See ItAbstractDataStreamerTest#testSameItemInsertUpdateRemove

    @ParameterizedTest
    @ValueSource(ints = {1, 2, 3})
    public void testSameItemInsertUpdateRemove(int pageSize) {
        RecordView<Tuple> view = defaultTable().recordView();
        CompletableFuture<Void> streamerFut;
        int key = 333000;

        try (var publisher = new SubmissionPublisher<DataStreamerItem<Tuple>>()) {
            streamerFut = view.streamData(publisher, DataStreamerOptions.builder().pageSize(pageSize).build());

            publisher.submit(DataStreamerItem.of(tuple(key, "foo")));
            publisher.submit(DataStreamerItem.removed(tupleKey(key)));
        }

        streamerFut.orTimeout(1, TimeUnit.SECONDS).join();

        assertNull(view.get(null, tupleKey(key)));
    }

Issue Links

• is related to: IGNITE-21521 Wrong update order in upsertAll and DataStreamer for a new key
• is related to: IGNITE-21401 DataStreamer data removal: improve RW_UPSERT_ALL to support delete
• links to: GitHub Pull Request #3713