[HUDI-4761] Test using spark listeners that guards any changes to DAG

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: tests-ci
• Story Points: 3
• Epic Link: Perf test env
• Assignee: sivabalan narayanan
• Reporter: Sagar Sumit
• Created: 01/Sep/22 07:18
• Updated: 10/Mar/23 01:03

Description

For major write operations and table services, we should guard the DAG by making use of spark listeners. Some basic things can be validated such as no. of stages, tasks, executor runtime, records written.