I am testing writing delta, OSS not databricks, data to Ozone FS since my company is looking to replace Hadoop if feasible. However, whenever I write delta table, the parquet files are writing, the delta log directory is created, but the json is never writing.
I am using the spark operator to submit a batch test job to write about 5mb of data.
Neither on the driver nor on the executor is there an error. The driver never finishes since the creation of the json hangs.
Code I used for testing spark operator and then I ran the pieces in the shell for testing. In the save path, update bucket and volume info for your data store.