Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-38852 Better Data Source V2 operator pushdown framework
  3. SPARK-38196

Reactor framework so as JDBC dialect could compile expression by self way

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0
    • SQL
    • None

    Description

      https://issues.apache.org/jira/browse/SPARK-37960 provides a new framework to represent catalyst expressions in DS V2 APIs.
      Because the framework translate all catalyst expressions to a unified SQL string and cannot keep compatibility between different JDBC database, the framework works not good.

      This PR reactor the framework so as JDBC dialect could compile expression by self way.
      First, The framework translate catalyst expressions to DS V2 expression.
      Second, The JDBC dialect could compile DS V2 expression to different SQL syntax.

      Attachments

        Activity

          People

            beliefer Jiaan Geng
            beliefer Jiaan Geng
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: