[SPARK-29794] Column level compression

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Anirudh Vyas
• Created: 07/Nov/19 17:08
• Updated: 16/Mar/20 22:54

Description

Currently in spark we do not have capability to specify different compressions for different columns, however this capability exists in parquet format for example.

 

Not sure if this has been opened before (I am sure it might have been but I cannot find it), hence opening a lane for potential improvement.