[SEDONA-405] Sedona driver Out of Memory on 1.4.1

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.1
• Fix Version/s: 1.5.0
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Jia Yu
• Created: 29/Sep/23 05:41
• Updated: 06/Oct/23 20:00
• Resolved: 06/Oct/23 20:00

Description

This was reported by Issue https://github.com/apache/sedona/issues/1040

 

The root cause is https://github.com/apache/sedona/pull/851 introduces Sedona Metric for RDD-based spatial join (nested loop join, left/right indexed join). This leads to OOM issue for RDD based join.

 

This issue also exists in Sedona SQL join (dynamic indexed join) for a long time but was not discovered in the past.

Issue Links

• links to: GitHub Pull Request #1041