[HUDI-5738] Config Simplification - ASF JIRA

Attach files

Attach Screenshot

Add vote

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Epic
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: 1.1.0
Component/s: configs
Labels:
None

Epic Name:
Config Simplification

Description

EPIC for simplifying configs and improving usability around configs in Hudi

Attachments

Hudi configuration parameters.xlsx
29/Feb/24 08:31
110 kB
Geser Dugarov

Issues in epic

quick-create-issue-for-epic-label

HUDI-5713	Add advanced property for configs	Closed	Lokesh Jain	Actions
HUDI-5723	Automate enum configs	Closed	Jonathan Vexler	Actions
HUDI-5739	Review configs	Closed	Ethan Guo	Actions
HUDI-5740	Refactor Deltastreamer and schema providers to use HoodieConfig/ConfigProperty	Closed	Lokesh Jain	Actions
HUDI-5742	Mark configs that are spark-sql compatible	Open	Unassigned	Actions
HUDI-5761	Create "Type" configs for current configs that take in classes	Open	Unassigned	Actions
HUDI-5762	Unify Config Types	Open	Unassigned	Actions
HUDI-5779	Annotate HoodieTableServiceManagerConfig for docs generation	Open	Shiyan Xu	Actions
HUDI-5780	Refactor Deltastreamer source configs to use HoodieConfig/ConfigProperty	Closed	Unassigned	Actions
HUDI-5781	Refactor other classes to use HoodieConfig/ConfigProperty	Closed	Unassigned	Actions
HUDI-5782	Tweak defaults and remove unnecessary configs after config review	Closed	Lokesh Jain	Actions
HUDI-5892	Code refactoring for config defaults and wiring	Open	Unassigned	Actions
HUDI-5893	Mark advanced configs	Closed	Ethan Guo	Actions
HUDI-5894	Revisit all parallelism configs	Closed	Ethan Guo	Actions
HUDI-5895	Simplify write configs for bootstrap	Open	Ethan Guo	Actions
HUDI-5896	Rename configs and make them backwards compatible	Closed	Unassigned	Actions
HUDI-5897	Improve config description	Open	Unassigned	Actions
HUDI-5898	Improve config structure for table services	Open	Unassigned	Actions
HUDI-5899	Make sure consistency check logic is not triggered by default	Closed	Ethan Guo	Actions
HUDI-5900	Clean up unused metadata configs	Closed	Lokesh Jain	Actions
HUDI-5925	Improve bootstrap parallelism	Closed	Ethan Guo	Actions
HUDI-5926	Improve cleaner parallelism	Closed	Ethan Guo	Actions
HUDI-5927	Improve parallelism of deleting invalid files	Closed	Ethan Guo	Actions
HUDI-5929	Automatically infer key generator type	Closed	Ethan Guo	Actions
HUDI-5934	Remove archival configs for MDT	Closed	Ethan Guo	Actions
HUDI-5954	Infer clean policy based on clean configs	Closed	Ethan Guo	Actions
HUDI-5959	Add docs for clean policy inference	Open	Shiyan Xu	Actions
HUDI-6062	Update LayoutOptimizationStrategy and ClusteringOperator to use standard enum notation	In Progress	Jonathan Vexler	Actions
HUDI-6075	Improve config generation script and docs	Closed	Ethan Guo	Actions
HUDI-6082	Mark advanced Flink configs	Patch Available	Ethan Guo	Actions
HUDI-6083	Fix advanced marking not taking effect for a few configs in DataSourceOptions	Open	Ethan Guo	Actions
HUDI-6112	Improve Doc generatiion to generate config tables for basic and advanced configs	Closed	Bhavani Sudha	Actions
HUDI-6179	Add Deltastreamer config group description	Open	Ethan Guo	Actions
HUDI-6180	Use ConfigProperty for Timestamp keygen configs	Open	Ethan Guo	Actions
HUDI-6511	Iterate recently added RLI based configs to mark them advanced as needed	Closed	Bhavani Sudha	Actions
HUDI-6593	Refactor deltastreamer commnad line config params to use HoodieConfig/ConfigProperty	Open	Bhavani Sudha	Actions
HUDI-6972	Fix redirection to individual config links	Closed	Bhavani Sudha	Actions
HUDI-6974	Cleanup config descriptions for consistent language and clarity	Open	Bhavani Sudha	Actions