[PHOENIX-6685] Change Data Capture - Populate Table / Topic Mappings

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.2.0
• Release Note: Added STREAMING_TOPIC_NAME to System.Catalog, which can be set as a property on CREATE and ALTER statements
• Assignee: Geoffrey Jacoby
• Reporter: Geoffrey Jacoby
• Created: 13/Apr/22 19:45
• Updated: 18/May/22 18:15
• Resolved: 18/May/22 18:15

Description

When creating a new table with CHANGE_DETECTION_ENABLED, or when altering an existing table to set this property to true, we should persist a mapping between the table being enabled, and the message broker topic which will carry its change messages.

This will allow both producers and consumers to use Phoenix metadata APIs to figure out which topic to produce or consume from.

In the CDC prototype my colleagues and I have been working with, this has been using a distinct system table we've manually populated. However, we may want to consider using System.Catalog linking rows instead, to make things more atomic. If table and topic will always be 1:1, we can also consider just adding an optional field to SYSTEM.CATALOG that views and tables can populate.

Issue Links

• links to: GitHub Pull Request #1441