Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.0.0-beta1
Description
- Embedded data streamer uses partition-based org.apache.ignite.internal.table.InternalTable#upsertAll(java.util.Collection<org.apache.ignite.internal.schema.BinaryRowEx>, int) method to upsert data, but client streamer uses upsertAll through public API
- Embedded streamer performs batching per partition, client streamer batches per node (connection)
Compare embedded and client streamer performance in different cases:
- 1 node, 4 nodes
- client connects to single node, client connects to all nodes
Check if using per-partition approach with internal upsertAll API improves client streamer performance.
UPDATE
IGNITE-21402 changed the logic to use per-partition batches:
- Compare client and embedded streamer performance
- Check if per-node is better for client
- Rename perNodeParallelOperations if necessary
Attachments
Issue Links
- is related to
-
IGNITE-21402 DataStreamer data removal: refactor client and embedded streamers to use the same internal API
- Resolved