[HUDI-7796] Gracefully cast file system instance in Avro writers

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.15.0, 1.0.0
• Labels: pull-request-available
• Story Points: 1
• Assignee: Ethan Guo (this is the old account; please use "yihua")
• Reporter: Ethan Guo (this is the old account; please use "yihua")
• Created: 25/May/24 06:35
• Updated: 05/Jun/24 00:57
• Resolved: 05/Jun/24 00:57

Description

When running tests in Trino with Hudi MDT enabled, the following line in HoodieAvroHFileWriter throws class cast exception, since Trino uses dependency injection to provide the Hadoop file system instance, which may skip the Hudi wrapper file system logic.

    this.fs = (HoodieWrapperFileSystem) this.file.getFileSystem(conf); 

Caused by: java.lang.ClassCastException: class io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper cannot be cast to class org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem (io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper and org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem are in unnamed module of loader 'app')
    at org.apache.hudi.io.hadoop.HoodieAvroHFileWriter.<init>(HoodieAvroHFileWriter.java:91)
    at org.apache.hudi.io.hadoop.HoodieAvroFileWriterFactory.newHFileFileWriter(HoodieAvroFileWriterFactory.java:108)
    at org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriterByFormat(HoodieFileWriterFactory.java:70)
    at org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriter(HoodieFileWriterFactory.java:53)
    at org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:108)
    at org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:77)
    at org.apache.hudi.io.CreateHandleFactory.create(CreateHandleFactory.java:45)
    at org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:101)
    at org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:44) 

Issue Links

• links to: GitHub Pull Request #11301
• links to: GitHub Pull Request #11304