[HUDI-5980] Add tests to guard against repeated dag trigger using spark event listeners

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Done
• Fix Version/s: 0.14.0
• Component/s: tests-ci
• Labels: pull-request-available
• Assignee: Lokesh Jain
• Reporter: sivabalan narayanan
• Created: 24/Mar/23 17:10
• Updated: 05/Oct/23 01:38
• Resolved: 05/Oct/23 01:38

Description

as of now, we don't have a good way to guard repeated dag trigger. all of our existing tests only checks for data. but w/ reconcile strategy, the extra files will be removed if some dag was repeated. So, we might need to add more tests to catch them if incase someone changes the dag in future. 

 

Eg test: https://github.com/apache/hudi/pull/8275/files

 

 

Issue Links

• links to: GitHub Pull Request #8641