[HUDI-96] Use Command line options instead of positional arguments when launching spark applications from various CLI commands

• Type: Task
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Component/s: cli
• Labels: new-to-hudi newbie pull-request-available sev:normal
• Epic Link: Improve Hudi CLI
• Assignee: Pratyaksh Sharma
• Reporter: Balaji Varadarajan
• Created: 09/Apr/19 21:12
• Updated: 21/Dec/23 15:33

Description

Hoodie CLI commands like compaction/rollback/repair/savepoints/parquet-import relies on launching a spark application to perform their operations (look at SparkMain.java). 

SparkMain (Look at SparkMain.main()) relies on positional arguments for passing  various CLI options. Instead we should define proper CLI options in SparkMain and use them (using Jcommander)  to improve readability and avoid accidental errors at call sites. For e.g : See com.uber.hoodie.utilities.HoodieCompactor

Issue Links

• is duplicated by: HUDI-2104 Add builder for hudi cli args
• links to: GitHub Pull Request #673
• links to: GitHub Pull Request #1174
• links to: GitHub Pull Request #1174