[FLINK-10287] Flink HA Persist Cancelled Job in Zookeeper

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.6.0
• Component/s: Runtime / Coordination
• Assignee: Unassigned
• Reporter: Sayat Satybaldiyev
• Created: 05/Sep/18 14:50
• Updated: 27/Mar/20 14:29
• Resolved: 27/Mar/20 14:29

Description

Flink HA persisted canceled job in Zookeeper, which makes HA mode quite fragile. In case JM get restarted, it tries to recover canceled job and after some time fails completely being not able to recover it. 

 

How to reproduce:

1. Have Flink HA 1.6 cluster
2. Cancel a running flink job
3. Observe that flink didn't remove ZK metadata.

ls /flink/flink_ns/jobgraphs/46d8d3555936c0d8e6b6ec21cc02bb11
[7f392fd9-cedc-4978-9186-1f54b98eeeb7]