[HUDI-781] Re-design test utilities

• Type: Test
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 0.7.0
• Component/s: Testing, tests-ci
• Labels: pull-request-available
• Assignee: Raymond Xu
• Reporter: Raymond Xu
• Created: 09/Apr/20 23:53
• Updated: 27/Mar/22 15:00
• Resolved: 13/Oct/20 00:47

Description

Test utility classes are to re-designed with considerations like

• Use more mockings
• Reduce spark context setup
• Improve/clean up data generator

An RFC would be preferred for illustrating the design work.

Issue Links

• is a child of: HUDI-779 [Umbrella] Unit tests improvements
• is blocked by: HUDI-811 Restructure test packages
• is depended upon by: HUDI-1249 [UMBRELLA] refactor tests for ease of development
• links to: GitHub Pull Request #1871
• links to: GitHub Pull Request #1997
• links to: GitHub Pull Request #2040

Sub-Tasks

• Parallelize CI testing to reduce CI wait time: Closed
• Identify functional tests that are convertible to unit tests with mocks: Closed
• Organize test utils methods and classes: Closed
• Use shared spark session provider: Closed