[SEDONA-28] Add WKB serializer in RDD and SQL API and let the user choose the SerDe

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Labels: pull-request-available
• Assignee: Netanel Malka
• Reporter: Netanel Malka
• Created: 29/Mar/21 21:01
• Updated: 04/Jan/23 22:33
• Resolved: 04/Jan/23 22:33

Description

Add WKB serializer as an optional serializer and use the old Serde as the default SerDe for both Core and SQL. The user should only use it if they use geometries that are currently not supported by the old Serde.
The user should be able to choose the serializer as follows

.config("spark.serializer", classOf[KryoSerializer].getName) // org.apache.spark.serializer.KryoSerializer
.config("spark.kryo.registrator", classOf[SedonaKryoRegistrator].getName)
.config("spark.serializer", classOf[KryoSerializer].getName) // org.apache.spark.serializer.KryoSerializer
.config("spark.kryo.registrator", classOf[SedonaWKBKryoRegistrator].getName)
 

Issue Links

• fixes: SEDONA-17 Replace geometry serializer in RDD API with the WKB serializer
• is caused by: SEDONA-17 Replace geometry serializer in RDD API with the WKB serializer
• is fixed by: SEDONA-207 Faster serialization/deserialization of geometry objects
• links to: GitHub Pull Request #516
• links to: GitHub Pull Request #544