[IGNITE-17748] Enrich InternalTable.scan API in order to support index scans

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-beta1
• Labels: ignite-3
• Epic Link: Ignite 3 transactions
• Assignee: Andrey Mashenkov
• Reporter: Alexander Lapin
• Created: 23/Sep/22 14:24
• Updated: 06/Dec/22 08:47
• Resolved: 04/Nov/22 11:13

Description

Motivation

SQL engine might specify index to use for scanning data along with some additional parameters like lower/upper bounds, including/excluding such bounds, columns to include etc. All in all we should enrich InternalTable scan api and provide corresponding data propagation logic. 

Definition of Done

1. InternalTable scan API has following method for both hash and sorted indexes scanning

Publisher<BinaryTuple> scan(int partId, @Nullable InternalTransaction tx, @NotNull UUID indexId, BinaryTuple key, BitSet columnsToInclude)

and following method for sorted index scanning

Publisher<BinaryTuple> scan(int partId, @Nullable InternalTransaction tx, @NotNull UUID indexId, @Nullable BinaryTuple leftBound, @Nullable BinaryTuple rightBound, int flags, BitSet columnsToInclude) 

Please check org.apache.ignite.internal.storage.index.SortedIndexStorage#scan for flags explanation, briefly

flags Control flags. {@link #GREATER} | {@link #LESS} by default. Other available values are {@link #GREATER_OR_EQUAL}, {@link #LESS_OR_EQUAL}.

2. Besides API itself corresponding scan-meta should be propagated to PartitionReplicaListener, so that some changes are also expected within ScanRetrieveBatchReplicaRequest. Please, pay attention that, that there's, probably, no sense in propagation same scan-meta within every ScanRetrieveBatchReplicaRequest, we might do it only wihtin initial request.

3. Proper index is chosen. See optional indexId param and proper method of either IndexStorage or specific SortedIndexStorage is used.

Implementation Notes

Mainly it's all specified in the section above. Seems that there's only one caveat left, however, non trivial one - BinaryRow to Binary tuple convertion, schema involved.

UPD:

As was discussed, indexId will be always specified so that tables internals will never select PK-index or any other by themselves, so that @Nullable UUID indexId is now @NotNull UUID indexId.

 

Besides API extension itself, it's required to build a bridge between https://issues.apache.org/jira/browse/IGNITE-17655 and internalTable.scan API, meaaning that it's required to create implementations for sql index interfaces introduced in 17655 that will propagate index scans to corresponding internalTable.scan API.

Issue Links

• causes: IGNITE-18090 Sql. Shutdown scan query executor on node stop.
• causes: IGNITE-18055 Sql. Support index scans with NULL conditions.
• causes: IGNITE-18056 Sql. Make RangeConditions accordant to index row type
• causes: IGNITE-18057 Sql. Fix index scan transactional consistency.
• is duplicated by: IGNITE-18055 Sql. Support index scans with NULL conditions.
• links to: GitHub pull request
• links to: GitHub Pull Request #1205
• links to: GitHub Pull Request #1311