[SPARK-38032] Upgrade Arrow version < 7.0.0 for Python UDF tests in SQL

• Type: Test
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: PySpark, SQL
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 26/Jan/22 04:53
• Updated: 12/Dec/22 18:10
• Resolved: 26/Jan/22 06:56

Description

We should better test latest PyArrow version. Now 6.0.1 is release but we're using < 5.0.0 for https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/IntegratedUDFTestUtils.scala

Issue Links

• links to: [Github] Pull Request #35331 (HyukjinKwon)