[SPARK-34479] Add zstandard codec to spark.sql.avro.compression.codec

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: SQL
• Assignee: Yuming Wang
• Reporter: Yuming Wang
• Created: 20/Feb/21 02:53
• Updated: 09/Sep/21 05:25
• Resolved: 27/Feb/21 18:32

Description

Avro add zstandard codec since AVRO-2195.

Issue Links

• is related to: AVRO-3060 Support ZSTD level and BufferPool options
• relates to: SPARK-34496 Upgrade ZSTD-JNI to 1.4.8-5 to API compatibility
• links to: [Github] Pull Request #31673 (wangyum)
• links to: [Github] Pull Request #32050 (williamhyun)
• links to: [Github] Pull Request #32050 (williamhyun)
• links to: [Github] Pull Request #33943 (viirya)