[HUDI-5894] Revisit all parallelism configs

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Story Points: 1
• Epic Link: Config Simplification
• Assignee: Ethan Guo
• Reporter: Ethan Guo
• Created: 07/Mar/23 23:57
• Updated: 18/Mar/23 20:54
• Resolved: 18/Mar/23 20:54

Description

See https://github.com/apache/hudi/pull/7912/files for relevant comments

Revisit the code logic to see if we can automatically derive better defaults if easy.

Configs in scope:

DELETE_ARCHIVED_INSTANT_PARALLELISM_VALUE

"hoodie.bootstrap.parallelism"

CLEANER_PARALLELISM_VALUE

SIMPLE_INDEX_PARALLELISM

ROLLBACK_PARALLELISM_VALUE

FINALIZE_WRITE_PARALLELISM_VALUE

MARKERS_DELETE_PARALLELISM_VALUE

Issue Links

• is fixed by: HUDI-5920 Improve documentation of parallelism configs
• is fixed by: HUDI-5925 Improve bootstrap parallelism
• is fixed by: HUDI-5926 Improve cleaner parallelism
• is fixed by: HUDI-5927 Improve parallelism of deleting invalid files