[PHOENIX-6354] Update to spark 3.0

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: spark-connector
• Assignee: Unassigned
• Reporter: Alejandro Anadon
• Created: 01/Feb/21 17:22
• Updated: 09/Feb/22 07:47
• Resolved: 08/Feb/22 09:00

Description

I am trying to use the phoenix spark connector with spark 3.0.

I tried to compile from trunk; but it is still configured with 2.4.0.

Is is not as easy (I am naive) as change the pon.xml from :

<spark.version>2.4.0</spark.version>

to

<spark.version>3.0.1</spark.version>

 

I found that in class :

org.apache.phoenix.spark.datasource.v2.PhoenixDataSource is still using the old DataSourcesV2 API; and I don't have the knowledge for updating it in a brand.

 

 

 

Issue Links

• duplicates: PHOENIX-6632 Migrate connectors to Spark-3