[FLINK-29916] Levels in Table Store may mistakenly ignore level 0 files when two files have the same sequence number

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: table-store-0.3.0, table-store-0.2.2
• Fix Version/s: table-store-0.3.0, table-store-0.2.2
• Component/s: Table Store
• Labels: pull-request-available
• Assignee: Caizhi Weng
• Reporter: Caizhi Weng
• Created: 07/Nov/22 09:30
• Updated: 08/Nov/22 12:20
• Resolved: 08/Nov/22 12:20

Description

Current constructor of Levels class contains the following code:

this.level0 = new TreeSet<>(Comparator.comparing(DataFileMeta::maxSequenceNumber).reversed());

However when two or more jobs writing the same bucket, they may produce files containing the same sequence number. If two files have the same maxSequenceNumber, one of them will be mistakenly ignored by TreeSet.

Issue Links

• links to: GitHub Pull Request #356
• links to: GitHub Pull Request #363