[FLINK-28910] CDC From Mysql To Hbase Bugs

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Resolved
• Component/s: Connectors / HBase
• Labels: pull-request-available stale-blocker
• Assignee: Unassigned
• Reporter: TE
• Created: 10/Aug/22 10:29
• Updated: 30/Jun/23 12:41
• Resolved: 30/Jun/23 12:30

Description

I use Flink for CDC from Mysql to Hbase.

The problem I encountered is that the Mysql record is updated (not deleted), but the record in hbase is deleted sometimes.

I tried to analyze the problem and found the reason as follows:

The update action of Mysql will be decomposed into delete + insert by Flink.
The Hbase connector uses a mutator to handle this set of actions.

However, if the order of these actions is not actively set, the processing of the mutator will not guarantee the order of execution.

Therefore, when the update of Mysql is triggered, it is possible that hbase actually performed the actions in the order of put + delete, resulting in the data being deleted.

Issue Links

• is superceded by: FLINK-32139 Data accidentally deleted and not deleted when upsert sink to hbase
• links to: GitHub Pull Request #20542