Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-38852 Better Data Source V2 operator pushdown framework
  3. SPARK-42131

Extract the function that construct the select statement for JDBC dialect.

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.5.0
    • SQL
    • None

    Description

      Currently, JDBCRDD uses fixed format for SELECT statement.

      val sqlText = options.prepareQuery +
            s"SELECT $columnList FROM ${options.tableOrQuery} $myTableSampleClause" +
            s" $myWhereClause $getGroupByClause $getOrderByClause $myLimitClause $myOffsetClause"
      

      But some databases have different syntax that uses different keyword or sort. For example, MS SQL Server uses keyword TOP to describe LIMIT clause or Top N.
      The LIMIT clause of MS SQL Server.

      SELECT TOP(1) Model, Color, Price  
            FROM dbo.Cars  
            WHERE Color = 'blue'
      

      The Top N of MS SQL Server.

      SELECT TOP(1) Model, Color, Price  
      FROM dbo.Cars  
      WHERE Color = 'blue'  
      ORDER BY Price ASC
      

      Attachments

        Activity

          People

            beliefer Jiaan Geng
            beliefer Jiaan Geng
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: