[PHOENIX-5435] Annotate HBase WALs with Phoenix Metadata

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.1.0, 4.16.0
• Assignee: Geoffrey Jacoby
• Reporter: Geoffrey Jacoby
• Created: 08/Aug/19 21:50
• Updated: 31/Dec/20 23:56
• Resolved: 31/Dec/20 23:56

Description

HBase write-ahead-logs (WALs) drive not only failure recovery, but HBase replication and some HBase backup frameworks. The WALs contain HBase-level metadata such as table and region, but lack Phoenix-level metadata. That means that it's quite difficult to build correct logic that needs to know about Phoenix-level constructs such as multi-tenancy, views, or indexes.

HBASE-22622 and HBASE-22623 add the capacity for coprocessors to annotate extra key/value pairs of metadata into the HBase WAL. We should have the option to annotate the tuple <tenant_id, table-or-view-name, timestamp>, or some hashed way to reconstruct that tuple into the WAL. It should have a feature toggle so operators who don't need it don't bear the slight extra storage cost.

Issue Links

• depends upon: HBASE-22622 WALKey Extended Attributes
• depends upon: HBASE-22623 Add RegionObserver coprocessor hook for preWALAppend
• is depended upon by: PHOENIX-5315 Cross cluster replication of the base table only should be sufficient
• requires: PHOENIX-6186 Store table metadata last modified timestamp in PTable / System.Catalog
• links to: GitHub Pull Request #913
• links to: GitHub Pull Request #1030