Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
None
Description
Stacktrace:
java.lang.IllegalArgumentException: Unknown file format :file:/Users/ethan/Work/data/hudi/metadata_test_ds_mor_continuous_4/.hoodie/metadata/files/files-0000_0-93-815_20220208164926830001.hfile at org.apache.hudi.common.table.TableSchemaResolver.getTableParquetSchemaFromDataFile(TableSchemaResolver.java:103) at org.apache.hudi.common.table.TableSchemaResolver.getTableAvroSchemaFromDataFile(TableSchemaResolver.java:119) at org.apache.hudi.common.table.TableSchemaResolver.hasOperationField(TableSchemaResolver.java:480) at org.apache.hudi.common.table.TableSchemaResolver.<init>(TableSchemaResolver.java:65) at org.apache.hudi.table.HoodieTable.validateSchema(HoodieTable.java:682) at org.apache.hudi.table.HoodieTable.validateUpsertSchema(HoodieTable.java:698) at org.apache.hudi.client.SparkRDDWriteClient.upsertPreppedRecords(SparkRDDWriteClient.java:173) at org.apache.hudi.metadata.SparkHoodieBackedTableMetadataWriter.commit(SparkHoodieBackedTableMetadataWriter.java:154) at org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.processAndCommit(HoodieBackedTableMetadataWriter.java:663) at org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.update(HoodieBackedTableMetadataWriter.java:675) at org.apache.hudi.client.BaseHoodieWriteClient.lambda$writeTableMetadata$0(BaseHoodieWriteClient.java:273) at org.apache.hudi.common.util.Option.ifPresent(Option.java:96) at org.apache.hudi.client.BaseHoodieWriteClient.writeTableMetadata(BaseHoodieWriteClient.java:273) at org.apache.hudi.client.BaseHoodieWriteClient.commit(BaseHoodieWriteClient.java:229) at org.apache.hudi.client.BaseHoodieWriteClient.commitStats(BaseHoodieWriteClient.java:199) at org.apache.hudi.client.SparkRDDWriteClient.commit(SparkRDDWriteClient.java:127) at org.apache.hudi.utilities.deltastreamer.DeltaSync.writeToSink(DeltaSync.java:609) at org.apache.hudi.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:329) at org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:652) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
full logs: https://gist.github.com/yihua/e00a1caddacbdc570b5b757049750f39
Attachments
Issue Links
- relates to
-
HUDI-3462 List of fixes to Metadata table after 0.10.1
- Closed
- links to