[SPARK-33213] Upgrade Apache Arrow to 2.0.0

• Type: Dependency upgrade
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.1
• Fix Version/s: 3.1.0
• Component/s: SQL
• Assignee: Chao Sun
• Reporter: Chao Sun
• Created: 21/Oct/20 20:59
• Updated: 12/Dec/22 18:11
• Resolved: 10/Nov/20 03:07

Description

Apache Arrow 2.0.0 has just been released. This proposes to upgrade Spark's Arrow dependency to use 2.0.0, from the current 1.0.1.

Issue Links

• is duplicated by: SPARK-34588 Support int64 buffer lengths in Java for pyspark Pandas UDF as buffer expanding
• links to: [Github] Pull Request #30306 (sunchao)