[SEDONA-349] Drop the support of typed RDDs

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: pull-request-available
• Assignee: Jia Yu
• Reporter: Jia Yu
• Created: 08/Aug/23 06:49
• Updated: 14/Sep/23 06:08

Description

Types RDDs in Sedona, including PointRDD/PolygonRDD/RectangleRDD/LineStringRDD were introduced in GeoSpark 0.1.

 

I propose to remove these RDDs.

1. These typed RDDs are not well designed. For example, they cannot handle GeometryCollections, Multi-XXX objects. They also do not support a dataset that has geometries with mixed types.

 

2. These code hasn't been touched for a very long time and most people use Sedona SQL API + Sedona generic RDD which is SpatialRDD.

 

3. The existence of these RDDs make the code maintenance hard especially it involves numerous CRS transform API in the tests and we plan to switch the lat/lon order to lon/lat.

 

4. We already have good replacement methods for these RDDs. Users can easily create a SpatialRDD using WKBReader, WKTReader, GeoJSONreader, ShapefileReader.

 

Due to the high volume of the legacy code, I will start with removing redundant constructors of these typedRDDs. In particular, those call analyze(), storageLevel() and crsTransform() inside.

Issue Links

• blocks: SEDONA-39 Fix the Lon/lat order issue in Sedona
• links to: GitHub Pull Request #954
• links to: GitHub Pull Request #1019