[SPARK-15816] SQL server based on Postgres protocol

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Component/s: SQL
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Reynold Xin
• Created: 08/Jun/16 06:47
• Updated: 21/May/19 04:13
• Resolved: 21/May/19 04:13

Description

At Spark Summit today this idea came up from a discussion: it would be great to investigate the possibility of implementing a new SQL server using Postgres' protocol, in lieu of Hive ThriftServer 2. I'm creating this ticket to track this idea, in case others have feedback.

This server can have a simpler architecture, and allows users to leverage a wide range of tools that are already available for Postgres (and many commercial database systems based on Postgres).

Some of the problems we'd need to figure out are:

1. What is the Postgres protocol? Is there an official documentation for it?
2. How difficult would it be to implement that protocol in Spark (JVM in particular).
3. How does data type mapping work?
4. How does system commands work? Would Spark need to support all of Postgres' commands?
5. Any restrictions in supporting nested data?