[HUDI-1869] Upgrading Spark3 To 3.1

• Type: Task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.10.0
• Component/s: spark
• Labels: pull-request-available
• Epic Link: Hudi Spark SQL
• Assignee: Yann Byron
• Reporter: pengzhiwei

Reviewers

Raymond Xu

• Created: 29/Apr/21 08:40
• Updated: 24/Feb/22 07:14
• Resolved: 03/Nov/21 10:57

Description

Spark 3.1 has changed some behavior of the internal class and interface for both spark-sql and spark-core module.

Currently hudi can't compile success under the spark 3.1. We need support sql support for spark 3.1.

Issue Links

• is related to: HUDI-2426 spark sql extensions breaks read.table from metastore
• links to: GitHub Pull Request #3580
• links to: GitHub Pull Request #3844