Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15816

SQL server based on Postgres protocol

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:

      Description

      At Spark Summit today this idea came up from a discussion: it would be great to investigate the possibility of implementing a new SQL server using Postgres' protocol, in lieu of Hive ThriftServer 2. I'm creating this ticket to track this idea, in case others have feedback.

      This server can have a simpler architecture, and allows users to leverage a wide range of tools that are already available for Postgres (and many commercial database systems based on Postgres).

      Some of the problems we'd need to figure out are:

      1. What is the Postgres protocol? Is there an official documentation for it?
      2. How difficult would it be to implement that protocol in Spark (JVM in particular).
      3. How does data type mapping work?
      4. How does system commands work? Would Spark need to support all of Postgres' commands?
      5. Any restrictions in supporting nested data?

        Attachments

        1. New_SQL_Server_for_Spark.pdf
          245 kB
          Takeshi Yamamuro

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rxin Reynold Xin
            • Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: