[PHOENIX-6067] (5.x) Global Secondary Index Parity with 4.x

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 5.1.0
• Assignee: Geoffrey Jacoby
• Reporter: Swaroopa Kadam
• Created: 10/Aug/20 19:59
• Updated: 10/Feb/21 10:00
• Resolved: 14/Sep/20 17:34

Description

A large number of indexing JIRAs were done for Phoenix 4.16 but were originally unable to be ported to 5.x because of the lack of HBase 2.x support for PHOENIX-5645, max lookback age. This was eventually fixed in HBASE-24321 and PHOENIX-5881. Because some JIRAs later than the missing ones were ported to 5.x, applying them all one-by-one and testing each intermediate version would be impractical.

This JIRA will import the 4.16 global index implementation into 5.1.0, then fix HBase API usage to conform to HBase 2.x standards and Phoenix's HBase 2.x compatibility shim between minor versions. (For example, max lookback age features aren't supported in 2.1 and 2.2 because they lack HBASE-24321, and incremental index validation will be deactivated when running against HBase 2.1, because of the lack of HBASE-22710 to use raw Filters.)

Issue Links

• causes: PHOENIX-6138 commons-lang in IndexScrutinyTool break dependency:analyze
• causes: PHOENIX-6305 Throttling decision does not take offheap memstore size into account
• fixes: PHOENIX-5287 Incorrect results for COUNT(1) on a table with GLOBAL INDEX
• incorporates: PHOENIX-5874 IndexTool does not set TTL on its log tables correctly
• incorporates: PHOENIX-5969 Read repair reduces the number of rows returned for LIMIT queries
• incorporates: PHOENIX-5734 IndexScrutinyTool should not report rows beyond maxLookBack age
• incorporates: PHOENIX-5735 IndexTool's inline verification should not verify rows beyond max lookback age
• incorporates: PHOENIX-5878 IndexTool should not fail jobs when the only errors are due to rows out of MaxLookback
• incorporates: PHOENIX-5896 Implement incremental rebuild along the failed regions in IndexTool
• incorporates: PHOENIX-5928 Index rebuilds without replaying data table mutations
• incorporates: PHOENIX-5951 IndexTool output logging for past-max-lookback rows should be configurable
• incorporates: PHOENIX-5957 Index rebuild should remove old index rows with higher timestamp
• is blocked by: PHOENIX-5881 Port MaxLookbackAge logic to 5.x
• links to: PR