[HUDI-5632] CLI commands using Spark fails to execute with hudi-cli-bundle

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.13.0
• Labels: pull-request-available
• Story Points: 2
• Assignee: Ethan Guo (this is the old account; please use "yihua")
• Reporter: Ethan Guo (this is the old account; please use "yihua")
• Created: 28/Jan/23 02:58
• Updated: 30/Jan/23 18:17
• Resolved: 30/Jan/23 18:03

Description

The following commands which require launching Spark cannot be executed in Hudi CLI shell with hudi-cli-bundle:

savepoint create --commit <latest-commit-timestamp>
downgrade table --toVersion 3
upgrade table --toVersion 5 

Output:

hudi:hudi_trips_cow->savepoint create --commit 20230127115839445
437425 [main] INFO  org.apache.hudi.common.table.timeline.HoodieActiveTimeline [] - Loaded instants upto : Option{val=[20230127115839445__commit__COMPLETED]}
438860 [Thread-7] INFO  org.apache.hudi.cli.utils.InputStreamConsumer [] - 23/01/27 12:15:18 WARN Utils: Your hostname, Ethans-MacBook-Pro.local resolves to a loopback address: 127.0.0.1; using 192.168.1.21 instead (on interface en0)
438862 [Thread-7] INFO  org.apache.hudi.cli.utils.InputStreamConsumer [] - 23/01/27 12:15:18 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
439171 [Thread-6] INFO  org.apache.hudi.cli.utils.InputStreamConsumer [] - Error: Failed to load org.apache.hudi.cli.commands.SparkMain: org/apache/hudi/exception/HoodieSavepointException
439175 [Thread-7] INFO  org.apache.hudi.cli.utils.InputStreamConsumer [] - 23/01/27 12:15:18 INFO ShutdownHookManager: Shutdown hook called
439176 [Thread-7] INFO  org.apache.hudi.cli.utils.InputStreamConsumer [] - 23/01/27 12:15:18 INFO ShutdownHookManager: Deleting directory /private/var/folders/60/wk8qzx310fd32b2dp7mhzvdc0000gn/T/spark-6e5631ce-26bf-4ceb-82cc-7ce77fcb177e
439217 [main] INFO  org.apache.hudi.common.table.HoodieTableMetaClient [] - Loading HoodieTableMetaClient from /Users/ethan/Work/tmp/20230127-test-cli-bundle/hudi_trips_cow
439218 [main] INFO  org.apache.hudi.common.table.HoodieTableConfig [] - Loading table properties from /Users/ethan/Work/tmp/20230127-test-cli-bundle/hudi_trips_cow/.hoodie/hoodie.properties
439221 [main] INFO  org.apache.hudi.common.table.HoodieTableMetaClient [] - Finished Loading Table of type COPY_ON_WRITE(version=1, baseFileFormat=PARQUET) from /Users/ethan/Work/tmp/20230127-test-cli-bundle/hudi_trips_cow
Failed: Could not create savepoint "20230127115839445". 

 

 

Issue Links

• links to: GitHub Pull Request #7790