[SEDONA-278] WKB geometry column in Delta Lake table not recognized as such in functions ST_GeomFromWKB or ST_AsText

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.4.0
• Labels: Binary WKB
• Environment: Databricks Runtime 12.1 with Apache Sedona 1.4.0
• Assignee: Unassigned
• Reporter: Quentin Mailhos
• Created: 09/May/23 16:13
• Updated: 04/Jun/23 01:43
• Resolved: 04/Jun/23 01:43

Description

After upgrading to Databricks DBR 12.1 with Apache Sedona 1.4.0, functions ST_GeomFromWKB and ST_AsText fail to read a Well-Known Binary (WKB) type column from a Delta Lake table, please see attached screenshot.

SQL error message is ambiguous:

[DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE] Cannot resolve "st_geomfromwkb(geom)" due to data type mismatch: parameter 1 requires ("STRING" or "BINARY") type, however, "geom" is of "BINARY" type.;

Spark error message as follows:

Caused by: org.apache.spark.sql.AnalysisException: Invalid Spark read type: expected optional group geom (LIST) { repeated group list

{ required int32 element (INTEGER(8,true)); }

} to be list type but found Some(BinaryType)

Workflow used to work just fine in Databricks DBR 9 LTS with Apache Sedona 1.1.0.