Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30375

Feature Parity between PostgreSQL and Spark (implementation-dependent behaviours)

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.0.0
    • None
    • SQL
    • None

    Description

      Among the feature parity between Spark/PostgreSQL (SPARK-27764), this umbrella holds issues for the behaviours that some parts of DBMS-like systems (e.g., PostgreSQL, Oracle, MySQL, ...) follow.

      Attachments

        Issue Links

        1.
        Type Casts: expression::type Sub-task Closed Unassigned Actions
        2.
        Support ARRAY(sub-SELECT) expressions Sub-task Reopened Unassigned Actions
        3.
        Data type: POINT(x, y) Sub-task Closed Unassigned Actions
        4.
        Built-in function: generate_series Sub-task Closed Unassigned Actions
        5.
        Add built-in Array Functions: array_append Sub-task Resolved Unassigned Actions
        6.
        Built-in SQL Function Support - WIDTH_BUCKET Sub-task Resolved Takeshi Yamamuro Actions
        7.
        Add the justify_days(), justify_hours() and justify_interval() functions Sub-task Resolved Kent Yao Actions
        8.
        Hyperbolic Functions Sub-task Resolved Tony Zhang Actions
        9.
        Difference of interval type conversion between SparkSQL and PostgreSQL Sub-task Resolved Unassigned Actions
        10.
        Add built-in Array Functions: array_fill Sub-task Resolved Unassigned Actions
        11.
        Add built-in Array Functions: array_dims Sub-task Resolved Unassigned Actions
        12.
        Support negative length at LEFT/RIGHT SQL functions Sub-task Resolved Unassigned Actions
        13.
        Add built-in String Functions: quote_literal Sub-task Resolved Unassigned Actions
        14.
        Data Type Formatting Functions: `to_number` Sub-task Resolved jiaan.geng Actions
        15.
        Adds `random()` sql function Sub-task Resolved Dongjoon Hyun Actions
        16.
        Add built-in Array Functions: array_ndims Sub-task Resolved Unassigned Actions
        17.
        Add built-in Array Functions: array_length Sub-task Resolved Unassigned Actions
        18.
        Add built-in Array Functions: array_append Sub-task Resolved Unassigned Actions
        19.
        Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in CREATE TABLE Sub-task Resolved Unassigned Actions
        20.
        Support [CREATE|DROP] RULE - define a new plan rewrite rule Sub-task Resolved Unassigned Actions
        21.
        Trim the string when cast string type to booleans Sub-task Resolved Kent Yao Actions
        22.
        interval `1. second` should be invalid as PostgreSQL Sub-task Resolved Kent Yao Actions
        23.
        Add built-in Aggregate Functions: string_agg Sub-task Resolved Unassigned Actions
        24.
        String Functions: Add support split_part Sub-task Resolved Unassigned Actions
        25.
        Binary String Functions: SHA functions Sub-task Resolved Unassigned Actions
        26.
        Array Functions: unnest Sub-task Resolved Unassigned Actions
        27.
        Add make_interval() function Sub-task Resolved Max Gekk Actions
        28.
        Support ALTER TABLE [relname] SET SCHEMA [dbname] Sub-task Resolved Unassigned Actions
        29.
        NULLS FIRST for DESC and NULLS LAST for ASC Sub-task Resolved Unassigned Actions
        30.
        Support truncate multiple tables Sub-task Resolved Unassigned Actions
        31.
        Support User-defined Types Sub-task Resolved Unassigned Actions
        32.
        String Functions: Enhance LPAD/RPAD function Sub-task Resolved Yuming Wang Actions
        33.
        Infinity, -Infinity, NaN should be recognized in a case insensitive manner Sub-task Resolved Dilip Biswal Actions
        34.
        Support SELECT statements in VALUES of INSERT INTO Sub-task Resolved Unassigned Actions
        35.
        String Functions: Add support other 4 REGEXP functions Sub-task Resolved Unassigned Actions
        36.
        Support IS OF (<type>) predicate Sub-task Resolved Unassigned Actions
        37.
        Array indexing is 1-based Sub-task Resolved Unassigned Actions
        38.
        Ordered-Set Aggregate Functions Sub-task Closed Unassigned Actions
        39.
        String Functions: SUBSTRING support regular expression Sub-task Resolved Unassigned Actions
        40.
        Support ANSI Aggregate Function: array_agg Sub-task Resolved Unassigned Actions
        41.
        String Functions: regexp_replace is not compatible Sub-task Open Unassigned Actions
        42.
        String Functions: Enhance TRIM function Sub-task Resolved Yuming Wang Actions
        43.
        String Functions: decode/encode can not accept 'escape' and 'hex' as charset Sub-task Open Unassigned Actions
        44.
        String Functions: Add support btrim Sub-task Resolved jiaan.geng Actions
        45.
        Trigonometric Functions Sub-task Resolved Unassigned Actions
        46.
        Built-in Mathematical Functions: SCALE Sub-task Resolved Unassigned Actions
        47.
        The LOG function using 10 as the base, but Spark using E Sub-task Resolved Unassigned Actions
        48.
        Add built-in Array Functions: array_upper Sub-task Resolved Unassigned Actions
        49.
        Add built-in Array Functions: element to array concatenation Sub-task Open Unassigned Actions
        50.
        Support more Postgres JSON functions Sub-task Open Unassigned Actions
        51.
        Data Type Formatting Functions: `to_char` Sub-task Resolved Daniel Actions
        52.
        Add built-in Array Functions: array_cat Sub-task Resolved Unassigned Actions
        53.
        Hypothetical-Set Aggregate Functions Sub-task Resolved Unassigned Actions
        54.
        String function: string_to_array Sub-task Resolved Unassigned Actions
        55.
        System Information Functions Sub-task Resolved Unassigned Actions
        56.
        Aggregate Functions for Statistics Sub-task Resolved Unassigned Actions
        57.
        Implement boolean aggregates(BOOL_AND, BOOL_OR and EVERY) Sub-task Resolved Kent Yao Actions
        58.
        Support for converting float/double to binary format Sub-task Resolved Unassigned Actions
        59.
        Fully support char and varchar types Sub-task Closed Unassigned Actions
        60.
        Support RETURNING cause Sub-task Resolved Unassigned Actions
        61.
        Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres/Redshift Sub-task Open Unassigned Actions
        62.
        Spark trunc() func does not support for number group as PostgreSQL Sub-task Open Unassigned Actions
        63.
        Implement bitwise integer aggregates(BIT_AND and BIT_OR) Sub-task Resolved Kent Yao Actions
        64.
        Implement ILIKE operator Sub-task Closed Unassigned Actions
        65.
        Concat for array in Spark SQL is not the one in PostgreSQL but array_cat Sub-task Resolved Unassigned Actions
        66.
        List all built-in UDFs have different names Sub-task Resolved Unassigned Actions
        67.
        Support SUBSTRING(str FROM n1 [FOR n2]) syntax Sub-task Resolved Zhu, Lipeng Actions
        68.
        substr returns different values Sub-task Resolved Unassigned Actions
        69.
        Support bit string types/literals Sub-task Resolved Unassigned Actions
        70.
        Support geometric types Sub-task Resolved Unassigned Actions
        71.
        Support TRIM(trimStr FROM str) syntax Sub-task Resolved Yuming Wang Actions
        72.
        SELECT DISTINCT ON ( expression [, ...] ) syntax Sub-task Resolved Unassigned Actions
        73.
        Missing escape_string_warning and standard_conforming_strings config Sub-task Resolved Unassigned Actions
        74.
        Bankers' rounding for double types Sub-task Resolved Unassigned Actions
        75.
        extract support interval type Sub-task Resolved Unassigned Actions
        76.
        Copy data between a file and a table Sub-task Resolved Unassigned Actions
        77.
        Return seconds with fraction from `date_part`/`extract` Sub-task Resolved Max Gekk Actions
        78.
        Implement more text pattern operators Sub-task Resolved Unassigned Actions
        79.
        Support field synonyms at `extract` Sub-task Resolved Max Gekk Actions
        80.
        Accept 'on' and 'off' as input for boolean data type Sub-task Resolved YoungGyu Chun Actions
        81.
        Missing some mathematical operators Sub-task Resolved Unassigned Actions
        82.
        Cast numeric to integral type need round Sub-task Open Unassigned Actions
        83.
        Support `dow`, `isodow` and `doy` at `extract()` Sub-task Resolved Max Gekk Actions
        84.
        Table inheritance Sub-task Resolved Unassigned Actions
        85.
        Support submit sql content only contains comments. Sub-task Open Unassigned Actions
        86.
        Support window frame ORDER BY i ROWS BETWEEN (('foo' < 'foobar')::integer) PRECEDING AND CURRENT ROW Sub-task Open Unassigned Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            maropu Takeshi Yamamuro

            Dates

              Created:
              Updated:

              Slack

                Issue deployment