[ARROW-8824] [Rust] [DataFusion] Implement new SQL parser

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Fix Version/s: 2.0.0
• Component/s: Rust, Rust - DataFusion
• External issue URL: https://github.com/apache/arrow/issues/24967
• Assignee: Andy Grove
• Reporter: Andy Grove
• Created: 16/May/20 16:55
• Updated: 11/Jan/23 08:02
• Resolved: 14/Jul/20 13:46

Description

We currently depend on the sqlparser crate that I originally created but has moved on since the version we use and that project is aiming to support multiple SQL dialects and I don't think it is appropriate for what we need in DataFusion.

I think it would be better to build a new SQL parser as part of the DataFusion crate so that we can more easily maintain it, and it can use Arrow as the native type system.

Another option would be to try and donate the sqlparser 0.2.x code base but there are a fair number of committers and it is probably easier just to implement it from scratch (without referencing the existing code).

 

Sub-Tasks

• [Rust] Implement SQL tokenizer: Closed
• [Rust] Implement SQL parser: Closed