[SPARK-45481] Introduce a mapper for parquet compression codecs

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Jiaan Geng
• Reporter: Jiaan Geng
• Created: 10/Oct/23 08:57
• Updated: 01/Nov/23 01:26
• Resolved: 27/Oct/23 02:49

Description

Currently, Spark supported all the parquet compression codecs, but the parquet supported compression codecs and spark supported are not completely one-on-one due to Spark introduce a fake compression codecs none.

There are a lot of magic strings copy from parquet compression codecs. This issue lead to developers need to manually maintain its consistency. It is easy to make mistakes and reduce development efficiency.

Issue Links

• links to: GitHub Pull Request #43308
• links to: GitHub Pull Request #43571
• links to: GitHub Pull Request #43604