[HUDI-4008] Delta Streamer does not respect hoodie.datasource.write.drop.partition.columns config

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.10.1
• Fix Version/s: 0.11.0
• Environment: AWS EMR 6.5 / HUDI v0.10.1
• Assignee: Unassigned
• Reporter: Istvan Darvas
• Created: 02/May/22 10:36
• Updated: 04/May/22 20:15
• Resolved: 04/May/22 20:15

Description

Hi Guys!

 

I have set this parameter in DeltaStreamer

    hoodie.datasource.write.drop.partition.columns=true

but parquet files which were writen by DeltaStreamer contain the partition culumns.

 

I Also have to say, SparkAPI respects this parameter. So what I wrote with spark.write that was OK

 

Darvi

 

Issue Links

• duplicates: HUDI-3748 Hudi fails to insert into a partitioned table when the partition column is dropped from the parquet schema