Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30375

Feature Parity between PostgreSQL and Spark (implementation-dependent behaviours)

    XMLWordPrintableJSON

    Details

    • Type: Umbrella
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None

      Description

      Among the feature parity between Spark/PostgreSQL (SPARK-27764), this umbrella holds issues for the behaviours that some parts of DBMS-like systems (e.g., PostgreSQL, Oracle, MySQL, ...) follow.

        Attachments

          Issue Links

          1.
          Type Casts: expression::type Sub-task Closed Unassigned
          2.
          Support ARRAY(sub-SELECT) expressions Sub-task Reopened Unassigned
          3.
          Data type: POINT(x, y) Sub-task Closed Unassigned
          4.
          Built-in function: generate_series Sub-task Closed Unassigned
          5.
          Add built-in Array Functions: array_append Sub-task Resolved Unassigned
          6.
          Built-in SQL Function Support - WIDTH_BUCKET Sub-task Resolved Takeshi Yamamuro
          7.
          Add the justify_days(), justify_hours() and justify_interval() functions Sub-task Resolved Kent Yao
          8.
          Hyperbolic Functions Sub-task Resolved Tony Zhang
          9.
          Difference of interval type conversion between SparkSQL and PostgreSQL Sub-task Resolved Unassigned
          10.
          Add built-in Array Functions: array_fill Sub-task Resolved Unassigned
          11.
          Add built-in Array Functions: array_dims Sub-task Resolved Unassigned
          12.
          Support negative length at LEFT/RIGHT SQL functions Sub-task Resolved Unassigned
          13.
          Add built-in String Functions: quote_literal Sub-task Resolved Unassigned
          14.
          Data Type Formatting Functions: `to_number` Sub-task Resolved Unassigned
          15.
          Adds `random()` sql function Sub-task Resolved Dongjoon Hyun
          16.
          Add built-in Array Functions: array_ndims Sub-task Resolved Unassigned
          17.
          Add built-in Array Functions: array_length Sub-task Resolved Unassigned
          18.
          Add built-in Array Functions: array_append Sub-task Resolved Unassigned
          19.
          Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in CREATE TABLE Sub-task Resolved Unassigned
          20.
          Support [CREATE|DROP] RULE - define a new plan rewrite rule Sub-task Resolved Unassigned
          21.
          Trim the string when cast string type to booleans Sub-task Resolved Kent Yao
          22.
          interval `1. second` should be invalid as PostgreSQL Sub-task Resolved Kent Yao
          23.
          Add built-in Aggregate Functions: string_agg Sub-task Resolved Unassigned
          24.
          String Functions: Add support split_part Sub-task Resolved Unassigned
          25.
          Binary String Functions: SHA functions Sub-task Resolved Unassigned
          26.
          Array Functions: unnest Sub-task Resolved Unassigned
          27.
          Add make_interval() function Sub-task Resolved Maxim Gekk
          28.
          Support ALTER TABLE [relname] SET SCHEMA [dbname] Sub-task Resolved Unassigned
          29.
          NULLS FIRST for DESC and NULLS LAST for ASC Sub-task Resolved Unassigned
          30.
          Support truncate multiple tables Sub-task Resolved Unassigned
          31.
          Support User-defined Types Sub-task Resolved Unassigned
          32.
          String Functions: Enhance LPAD/RPAD function Sub-task Resolved Yuming Wang
          33.
          Infinity, -Infinity, NaN should be recognized in a case insensitive manner Sub-task Resolved Dilip Biswal
          34.
          Support SELECT statements in VALUES of INSERT INTO Sub-task Resolved Unassigned
          35.
          String Functions: Add support other 4 REGEXP functions Sub-task Resolved Unassigned
          36.
          Support IS OF (<type>) predicate Sub-task Resolved Unassigned
          37.
          Array indexing is 1-based Sub-task Resolved Unassigned
          38.
          Ordered-Set Aggregate Functions Sub-task Closed Unassigned
          39.
          String Functions: SUBSTRING support regular expression Sub-task Resolved Unassigned
          40.
          Add built-in Aggregate Function: array_agg Sub-task Open Unassigned
          41.
          String Functions: regexp_replace is not compatible Sub-task Open Unassigned
          42.
          String Functions: Enhance TRIM function Sub-task Resolved Yuming Wang
          43.
          String Functions: decode/encode can not accept 'escape' and 'hex' as charset Sub-task Open Unassigned
          44.
          String Functions: Add support btrim Sub-task Open Unassigned
          45.
          Trigonometric Functions Sub-task Resolved Unassigned
          46.
          Built-in Mathematical Functions: SCALE Sub-task Resolved Unassigned
          47.
          The LOG function using 10 as the base, but Spark using E Sub-task Resolved Unassigned
          48.
          Add built-in Array Functions: array_upper Sub-task Resolved Unassigned
          49.
          Add built-in Array Functions: element to array concatenation Sub-task Open Unassigned
          50.
          Support more Postgres JSON functions Sub-task Open Unassigned
          51.
          Data Type Formatting Functions: `to_char` Sub-task Resolved Unassigned
          52.
          Add built-in Array Functions: array_cat Sub-task Resolved Unassigned
          53.
          Hypothetical-Set Aggregate Functions Sub-task Resolved Unassigned
          54.
          String function: string_to_array Sub-task Resolved Unassigned
          55.
          System Information Functions Sub-task Resolved Unassigned
          56.
          Aggregate Functions for Statistics Sub-task Resolved Unassigned
          57.
          Implement boolean aggregates(BOOL_AND, BOOL_OR and EVERY) Sub-task Resolved Kent Yao
          58.
          Support for converting float/double to binary format Sub-task Resolved Unassigned
          59.
          Fully support char and varchar types Sub-task Closed Unassigned
          60.
          Support RETURNING cause Sub-task Resolved Unassigned
          61.
          Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres/Redshift Sub-task Open Unassigned
          62.
          Spark trunc() func does not support for number group as PostgreSQL Sub-task Open Unassigned
          63.
          Implement bitwise integer aggregates(BIT_AND and BIT_OR) Sub-task Resolved Kent Yao
          64.
          Implement ILIKE operator Sub-task Closed Unassigned
          65.
          Concat for array in Spark SQL is not the one in PostgreSQL but array_cat Sub-task Resolved Unassigned
          66.
          List all built-in UDFs have different names Sub-task Resolved Unassigned
          67.
          Support SUBSTRING(str FROM n1 [FOR n2]) syntax Sub-task Resolved Zhu, Lipeng
          68.
          substr returns different values Sub-task Resolved Unassigned
          69.
          Support bit string types/literals Sub-task Resolved Unassigned
          70.
          Support geometric types Sub-task Resolved Unassigned
          71.
          Support TRIM(trimStr FROM str) syntax Sub-task Resolved Yuming Wang
          72.
          SELECT DISTINCT ON ( expression [, ...] ) syntax Sub-task Resolved Unassigned
          73.
          Missing escape_string_warning and standard_conforming_strings config Sub-task Resolved Unassigned
          74.
          Bankers' rounding for double types Sub-task Resolved Unassigned
          75.
          extract support interval type Sub-task Resolved Unassigned
          76.
          Copy data between a file and a table Sub-task Resolved Unassigned
          77.
          Return seconds with fraction from `date_part`/`extract` Sub-task Resolved Maxim Gekk
          78.
          Implement more text pattern operators Sub-task Resolved Unassigned
          79.
          Support field synonyms at `extract` Sub-task Resolved Maxim Gekk
          80.
          Accept 'on' and 'off' as input for boolean data type Sub-task Resolved YoungGyu Chun
          81.
          Missing some mathematical operators Sub-task Resolved Unassigned
          82.
          Cast numeric to integral type need round Sub-task Open Unassigned
          83.
          Support `dow`, `isodow` and `doy` at `extract()` Sub-task Resolved Maxim Gekk
          84.
          Table inheritance Sub-task Resolved Unassigned
          85.
          Support submit sql content only contains comments. Sub-task Open Unassigned

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                maropu Takeshi Yamamuro
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: