[PHOENIX-7217] MaxLookback and TTL improvements and fixes

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.2.0
• Fix Version/s: 5.3.0
• Assignee: Viraj Jasani
• Reporter: Viraj Jasani
• Created: 16/Feb/24 02:08
• Updated: 16/Apr/24 05:53

Description

Some improvements/fixes for Phoenix Table MaxLookback and TTL feature:

• Use overridden value of maxLookback ms while confirming whether region level compaction is required
• Use map of table with map of ColumnFamily to MaxLookback value for the override API
• For the entire row version that is within max lookback window, avoid extra region level compaction
• Support DeleteFamilyVersion marker with DeleteFamily markers such that DeleteFamilyVersion marker can mask only Put cells with the same timestamp. This requires changes to how we build and consume the column list for all cells that are outside the maxLookback window. The changes are required in both phoenix and hbase level compaction. We need to retain DeleteFamilyVersion markers as they can be used to perform masking of old cells (inside or outside of TTL window). The combination of DeleteFamily and DeleteFamilyVersion markers need more tests.

Issue Links

• is related to: PHOENIX-6888 Fixing TTL and Max Lookback Issues for Phoenix Tables
• links to: GitHub Pull Request #1826