[SPARK-45758] Introduce a mapper for hadoop compression codecs

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Resolved
• Affects Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Jiaan Geng
• Reporter: Jiaan Geng
• Created: 01/Nov/23 12:54
• Updated: 06/Nov/23 10:07
• Resolved: 06/Nov/23 10:07

Description

Currently, Spark supported partial Hadoop compression codecs, but the Hadoop supported compression codecs and spark supported are not completely one-on-one due to Spark introduce two fake compression codecs none and uncompress.

There are a lot of magic strings copy from Hadoop compression codecs. This issue lead to developers need to manually maintain its consistency. It is easy to make mistakes and reduce development efficiency.

Issue Links

• links to: GitHub Pull Request #43620