Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27764

Feature Parity between PostgreSQL and Spark

    XMLWordPrintableJSON

    Details

    • Type: Umbrella
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None

      Description

      PostgreSQL is one of the most advanced open source databases. This umbrella Jira is trying to track the missing features and bugs. 

       

        Attachments

          Issue Links

          1.
          Type Casts: expression::type Sub-task Closed Unassigned
          2.
          Support ARRAY(sub-SELECT) expressions Sub-task In Progress Unassigned
          3.
          Implement bitwise integer aggregates(BIT_AND and BIT_OR) Sub-task Resolved Kent Yao
          4.
          Handling of sublinks within outer-level aggregates. Sub-task Open Unassigned
          5.
          Data type: POINT(x, y) Sub-task Closed Unassigned
          6.
          Built-in function: generate_series Sub-task Closed Unassigned
          7.
          Infinity, -Infinity, NaN should be recognized in a case insensitive manner Sub-task Resolved Dilip Biswal
          8.
          List all cases that PostgreSQL throws an exception but Spark SQL is NULL Sub-task Open Unassigned
          9.
          Add built-in Array Functions: array_append Sub-task Open Unassigned
          10.
          support statement using nested joins Sub-task Reopened Unassigned
          11.
          Implement boolean aggregates(BOOL_AND, BOOL_OR and EVERY) Sub-task Resolved Kent Yao
          12.
          List all built-in UDFs have different names Sub-task Open Unassigned
          13.
          ANSI SQL: NTH_VALUE function Sub-task Open Unassigned
          14.
          ANSI SQL: LATERAL derived table(T491) Sub-task Closed Unassigned
          15.
          Support 4 date operators(date + integer, integer + date, date - integer and date - date) Sub-task Resolved Yuming Wang
          16.
          Built-in SQL Function Support - WIDTH_BUCKET Sub-task Resolved Unassigned
          17.
          Support ANSI SQL Boolean-Predicate syntax Sub-task Resolved jiaan.geng
          18.
          Add built-in Aggregate Function: array_agg Sub-task Open Unassigned
          19.
          Accept 'on' and 'off' as input for boolean data type Sub-task Resolved YoungGyu Chun
          20.
          Support INTERVAL ... HOUR TO SECOND syntax Sub-task Resolved Zhu, Lipeng
          21.
          Support Aggregate Expressions with filter Sub-task Open Unassigned
          22.
          Interval type conversion syntax support Sub-task Resolved Zhu, Lipeng
          23.
          Support SUBSTRING(str FROM n1 [FOR n2]) syntax Sub-task Resolved Zhu, Lipeng
          24.
          String Functions: regexp_replace is not compatible Sub-task Open Unassigned
          25.
          Add built-in Aggregate Functions: string_agg Sub-task Open Unassigned
          26.
          Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres/Redshift Sub-task Open Unassigned
          27.
          Ordered-Set Aggregate Functions Sub-task Closed Unassigned
          28.
          Support POSIX Regular Expressions Sub-task Open Unassigned
          29.
          Support WITH clause in subqueries Sub-task Resolved Peter Toth
          30.
          Support WITH clause column aliases Sub-task Resolved Peter Toth
          31.
          Cast numeric to integral type need round Sub-task Open Unassigned
          32.
          Support ORDER BY ... USING syntax Sub-task Open Unassigned
          33.
          Support negative length at LEFT/RIGHT SQL functions Sub-task Open Unassigned
          34.
          `lag` second argument must be a literal Sub-task Open Unassigned
          35.
          Enhance DATE_TRUNC Sub-task Resolved Maxim Gekk
          36.
          Support for converting float/double to binary format Sub-task Open Unassigned
          37.
          Trim the string when cast string type to other types Sub-task Resolved Unassigned
          38.
          Missing some mathematical operators Sub-task Resolved Unassigned
          39.
          String concatenation should low priority than other operators Sub-task In Progress Unassigned
          40.
          Add built-in String Functions: quote_literal Sub-task Open Unassigned
          41.
          Binary String Functions: SHA functions Sub-task Open Unassigned
          42.
          ANSI SQL: Character literals Sub-task Resolved Unassigned
          43.
          Float/Double type can not accept some special inputs Sub-task Resolved Unassigned
          44.
          Order by does not accept a call to rank() Sub-task Open Unassigned
          45.
          ntile only accepting positive (>0) values Sub-task Open Unassigned
          46.
          String Functions: Add support other 4 REGEXP functions Sub-task Open Unassigned
          47.
          String Functions: Enhance TRIM function Sub-task Resolved Yuming Wang
          48.
          String Functions: SUBSTRING support regular expression Sub-task Open Unassigned
          49.
          ANSI SQL: OVERLAY function(T312) Sub-task Resolved jiaan.geng
          50.
          String Functions: Add support split_part Sub-task Open Unassigned
          51.
          Support ANSI SQL: LIKE ... ESCAPE syntax Sub-task In Progress Unassigned
          52.
          Adds `random()` sql function Sub-task Resolved Dongjoon Hyun
          53.
          String Functions: Enhance LPAD/RPAD function Sub-task Resolved Yuming Wang
          54.
          String Functions: decode/encode can not accept 'escape' and 'hex' as charset Sub-task Open Unassigned
          55.
          String Functions: Add support btrim Sub-task Open Unassigned
          56.
          Support IS OF (<type>) predicate Sub-task Open Unassigned
          57.
          Trigonometric Functions Sub-task Resolved Unassigned
          58.
          Support Interval comparison Sub-task Open Unassigned
          59.
          Hyperbolic Functions Sub-task Resolved Tony Zhang
          60.
          Data Type Formatting Functions: `to_number` Sub-task Open Unassigned
          61.
          Date type can not accept special values Sub-task Resolved Maxim Gekk
          62.
          Date/Timestamp type have different low value and high value with Spark Sub-task Open Unassigned
          63.
          Improved VALUES support Sub-task Open Unassigned
          64.
          Support TRIM(trimStr FROM str) syntax Sub-task Resolved Yuming Wang
          65.
          Support RETURNING cause Sub-task Open Unassigned
          66.
          Difference of interval type conversion between SparkSQL and PostgreSQL Sub-task Resolved Unassigned
          67.
          Fix substitution order of nested WITH clauses Sub-task Resolved Peter Toth
          68.
          Date/Time Output Styles and Date Order Conventions Sub-task Open Unassigned
          69.
          ANSI SQL: Recursive query Sub-task In Progress Unassigned
          70.
          Handling outer links in CTE subquery expressions Sub-task Open Unassigned
          71.
          Fully support char and varchar types Sub-task Closed Unassigned
          72.
          Evaluation of multiple CTE uses Sub-task Open Unassigned
          73.
          Decimal can not accept NaN as input Sub-task Open Unassigned
          74.
          Decimal precision issue Sub-task Resolved Unassigned
          75.
          Built-in Mathematical Functions: SCALE Sub-task Open Unassigned
          76.
          Decimal can only support precision up to 38 Sub-task Open Unassigned
          77.
          SELECT INTO syntax Sub-task Open Unassigned
          78.
          DIV support decimal type Sub-task Resolved Marco Gaido
          79.
          The LOG function using 10 as the base, but Spark using E Sub-task Open Unassigned
          80.
          Array Functions: unnest Sub-task Open Unassigned
          81.
          Support ANSI SQL:SIMILAR TO ... ESCAPE syntax Sub-task Open Unassigned
          82.
          Support ANSI SQL:member predicate Sub-task Open Unassigned
          83.
          ANSI SQL: Top-level <result offset clause> in <query expression> Sub-task Open Unassigned
          84.
          Add built-in Array Functions: array_upper Sub-task Resolved Unassigned
          85.
          NULLS FIRST for DESC and NULLS LAST for ASC Sub-task Resolved Unassigned
          86.
          Fully support correlation names in the FROM clause Sub-task Open Unassigned
          87.
          Correlated scalar subqueries must be aggregated Sub-task Resolved Unassigned
          88.
          SELECT DISTINCT ON ( expression [, ...] ) syntax Sub-task Open Unassigned
          89.
          Division operator support integral division Sub-task Resolved Yuming Wang
          90.
          Add more Date/Time Operators Sub-task Open Unassigned
          91.
          SELECT FROM syntax Sub-task Open Unassigned
          92.
          Add built-in Array Functions: element to array concatenation Sub-task Open Unassigned
          93.
          Array indexing is 1-based Sub-task Open Unassigned
          94.
          Support typed interval expression Sub-task Resolved Yuming Wang
          95.
          Join with USING caluse doesn't hide original tables Sub-task Open Unassigned
          96.
          Use Java 8 API to add months to dates Sub-task Resolved Maxim Gekk
          97.
          ANSI SQL: Unicode escapes in literals Sub-task Open Unassigned
          98.
          ANSI SQL(E021-05): OCTET_LENGTH function Sub-task Resolved Unassigned
          99.
          ANSI SQL: OVERLAY function support byte array(T312) Sub-task Resolved jiaan.geng
          100.
          Date/Time Functions: date_part for intervals Sub-task Resolved Maxim Gekk
          101.
          Spark `exclude` always expecting `()` Sub-task Open Unassigned
          102.
          Support more Postgres JSON functions Sub-task Open Unassigned
          103.
          SQL Datetime util function being casted to double instead of timestamp Sub-task Reopened Unassigned
          104.
          Add `make_date` function Sub-task Resolved Maxim Gekk
          105.
          Support `TIME` type in Spark Sub-task Open Unassigned
          106.
          Support accepting the interval keyword in the schema string Sub-task Resolved Yuming Wang
          107.
          Different format when casting interval type to string type Sub-task Open Unassigned
          108.
          Implement ILIKE operator Sub-task Closed Unassigned
          109.
          Missing escape_string_warning and standard_conforming_strings config Sub-task Open Unassigned
          110.
          Pad Decimal numbers with trailing zeros to the scale of the column Sub-task Open Unassigned
          111.
          substr returns different values Sub-task Open Unassigned
          112.
          Support recursive view syntax Sub-task Resolved Unassigned
          113.
          Date/Time Functions: make_timestamp Sub-task Resolved Maxim Gekk
          114.
          Formatting dates with negative years Sub-task Resolved Maxim Gekk
          115.
          adds support for `filter` clause Sub-task Open Unassigned
          116.
          frame bound must be a literal Sub-task Open Unassigned
          117.
          Support empty select list Sub-task Open Unassigned
          118.
          Recognize interval as a numeric type Sub-task Open Unassigned
          119.
          not handling usage of group function and window function at some conditions Sub-task Open Unassigned
          120.
          Support for range frame+row frame in the same query Sub-task Open Unassigned
          121.
          Data Type Formatting Functions: `to_char` Sub-task Open Unassigned
          122.
          subqueries must be aggregated before hand Sub-task Resolved Unassigned
          123.
          window functions should not be allowed in window definitions Sub-task Resolved Unassigned
          124.
          ANSI SQL: unique predicate Sub-task Open Unassigned
          125.
          Support `dow`, `isodow` and `doy` at `extract()` Sub-task Resolved Maxim Gekk
          126.
          Throw an error on window redefinition Sub-task Open Unassigned
          127.
          Allow usage of `count` only for parameterless aggregate function Sub-task Open Unassigned
          128.
          Adds support to `groups` unit type in window clauses Sub-task Open Unassigned
          129.
          Support `millennium`, `century` and `decade` at `extract()` Sub-task Resolved Maxim Gekk
          130.
          Hypothetical-Set Aggregate Functions Sub-task Resolved Unassigned
          131.
          Aggregate Functions for Statistics Sub-task Resolved Unassigned
          132.
          ORDER BY in aggregate function Sub-task Open Unassigned
          133.
          System Information Functions Sub-task Resolved Unassigned
          134.
          Spark should support select <columnname> into <table> from <table> where <condition> as PostgreSQL supports Sub-task Resolved Unassigned
          135.
          ANSI SQL: Collation Support Sub-task Open Unassigned
          136.
          Support `epoch`, `isoyear`, `milliseconds` and `microseconds` at `extract()` Sub-task Resolved Maxim Gekk
          137.
          Date/Time Functions: date_part for timestamps Sub-task Resolved Maxim Gekk
          138.
          Support field synonyms at `extract` Sub-task Resolved Maxim Gekk
          139.
          Support limit on recursive queries Sub-task Resolved Unassigned
          140.
          [UDF] Supports for alter, rename, owner change should be supported in Spark Sub-task Open Unassigned
          141.
          Implement more text pattern operators Sub-task Open Unassigned
          142.
          Table inheritance Sub-task Resolved Unassigned
          143.
          ANSI SQL: Bracketed comments Sub-task Open Unassigned
          144.
          ANSI SQL: delete and update does not support in Spark Sub-task Resolved Unassigned
          145.
          char data type should support in Spark Sub-task Open Unassigned
          146.
          Add TimeType to Catalyst Sub-task Open Unassigned
          147.
          Support special date/timestamp values `infinity`/`-infinity` Sub-task Open Unassigned
          148.
          Timestamp type can not accept special values Sub-task Resolved Maxim Gekk
          149.
          Ignore case while resolving time zones Sub-task Open Unassigned
          150.
          Support seconds precision by the timestamp type Sub-task Open Unassigned
          151.
          String Constants with C-style Escapes Sub-task Open Unassigned
          152.
          Test both ANSI mode and Spark mode Sub-task Open Unassigned
          153.
          DEFAULT option is not supported in Spark Sub-task Open Unassigned
          154.
          Support interval field values with fractional parts Sub-task Open Unassigned
          155.
          Return null from `date_part()` for the null `field` Sub-task Resolved Maxim Gekk
          156.
          Return seconds with fraction from `date_part`/`extract` Sub-task Resolved Maxim Gekk
          157.
          Make casting strings to intervals case insensitive Sub-task Resolved Maxim Gekk
          158.
          Support timestamps subtraction Sub-task Resolved Maxim Gekk
          159.
          Support date and timestamp subtraction Sub-task Resolved Unassigned
          160.
          Accept strings without `interval` prefix in casting to intervals Sub-task Resolved Maxim Gekk
          161.
          Interval strings without explicit unit markings Sub-task Open Unassigned
          162.
          Support CURRENT_TIME and LOCALTIME when ANSI mode enabled Sub-task Open Unassigned
          163.
          Support interval values without INTERVAL clauses Sub-task Resolved Takeshi Yamamuro
          164.
          Support LOCALTIMESTAMP when ANSI mode enabled Sub-task Resolved Unassigned
          165.
          Forbid reserved keywords as identifiers when ANSI mode is on Sub-task Resolved Takeshi Yamamuro
          166.
          define reserved keywords after SQL standard Sub-task Resolved Takeshi Yamamuro
          167.
          Support option to throw exception if overflow occurs during Decimal arithmetic Sub-task Resolved Marco Gaido
          168.
          The exact math method should be called only when there is a corresponding function in Math Sub-task Resolved Gengliang Wang
          169.
          Throw exception on overflow for integers Sub-task Resolved Marco Gaido
          170.
          Copy data between a file and a table Sub-task Open Unassigned
          171.
          Construct intervals from the `millenniums`, `centuries` or `decades` units Sub-task Open Unassigned
          172.
          Support synonyms for interval units Sub-task Open Unassigned
          173.
          Make `INTERVAL` values comparable Sub-task Resolved Unassigned
          174.
          Support writing `INTERVAL` type to datasource table Sub-task Open Unassigned
          175.
          Interval output styles Sub-task Open Unassigned
          176.
          Support interval literal with negative sign `-` Sub-task Resolved Maxim Gekk
          177.
          Support `*` and `/` operators for intervals Sub-task Resolved Maxim Gekk
          178.
          Support the optional prefix `@` in interval strings Sub-task Open Unassigned
          179.
          Support `ago` in interval strings Sub-task Open Unassigned
          180.
          Add the justify_days(), justify_hours() and justify_interval() functions Sub-task Resolved Kent Yao
          181.
          Default year-month units Sub-task Open Unassigned
          182.
          Add make_interval() function Sub-task Resolved Maxim Gekk
          183.
          Support ISO 8601 format for intervals Sub-task Open Unassigned
          184.
          Precision of the interval type Sub-task Open Unassigned
          185.
          Support syntax for zero interval Sub-task Open Unassigned
          186.
          String function: string_to_array Sub-task Open Unassigned
          187.
          Unordered interval units Sub-task Open Unassigned
          188.
          Thrift in some cases can't parse string to date Sub-task Open Unassigned
          189.
          Spark should work as PostgreSQL when using + Operator Sub-task Open Unassigned
          190.
          extract support interval type Sub-task Open Unassigned
          191.
          NOT NULL is not supported in Spark Sub-task Open Unassigned
          192.
          Support ANSI SQL CREATE SEQUENCE Sub-task Open Unassigned
          193.
          Support ALTER TABLE [relname] SET SCHEMA [dbname] Sub-task Open Unassigned
          194.
          Can't parse '11:00 BST' or '2000-10-19 10:23:54+01' signatures to timestamp Sub-task Open Unassigned
          195.
          Spark handles 'NaN' as 0 in sums Sub-task Open Unassigned
          196.
          Real data type is not supported in Spark SQL which is supporting in postgresql Sub-task Open Unassigned
          197.
          Support data insertion in a different order if you wish or even omit some columns in spark sql also like postgresql Sub-task Open Unassigned
          198.
          Bankers' rounding for double types Sub-task Open Unassigned
          199.
          Forcibly create a temporary view in CREATE VIEW if referencing a temporary view Sub-task Open Unassigned
          200.
          Support typed integer literal expression Sub-task Resolved Kent Yao
          201.
          Support cascaded syntax in CREATE SCHEMA Sub-task Open Unassigned
          202.
          Discard a NULL constant in LIMIT Sub-task Open Unassigned
          203.
          Make COLUMN optional in ALTER TABLE Sub-task Open Unassigned
          204.
          Not allowed to create a permanent view by referencing a temporary view in EXISTS Sub-task Resolved Terry Kim
          205.
          Support COMMENT ON syntax Sub-task Open Unassigned
          206.
          Some queries with divisions in SQL windows are failling in Thrift Sub-task Open Unassigned
          207.
          Support geometric types Sub-task Open Unassigned
          208.
          Support divide/multiply for interval types Sub-task Open Unassigned
          209.
          Dropping columns and changing column names/types are prohibited in VIEW definition Sub-task Open Unassigned
          210.
          Support bit string types/literals Sub-task Open Unassigned
          211.
          Support sum with interval type values Sub-task Resolved Kent Yao
          212.
          Make interval type camparable and orderable Sub-task Resolved Kent Yao
          213.
          Different answers in aggregates of multiple grouping sets Sub-task Open Unassigned
          214.
          Support average with interval type values Sub-task Resolved Kent Yao
          215.
          Support grouping function with multiple arguments Sub-task Open Unassigned
          216.
          Different answers in nested aggregates with window functions Sub-task Open Unassigned
          217.
          Support nested grouping sets Sub-task Open Unassigned
          218.
          Different answers when empty input given in GROUPING SETS Sub-task Open Unassigned
          219.
          Resolve group-by columns with functional dependencies Sub-task Open Unassigned
          220.
          Support Interval Unit Abbreviations in Interval Literals Sub-task Open Unassigned
          221.
          Support grouping() in GROUP BY without GroupingSets/Cube/Rollup Sub-task Open Unassigned
          222.
          Support the combinations of grouping operations Sub-task Open Unassigned
          223.
          Support more expressive forms in GroupingSets/Cube/Rollup Sub-task Open Unassigned
          224.
          Support an empty grouping expression Sub-task Open Unassigned
          225.
          Spark trunc() func does not support for number group as PostgreSQL Sub-task Open Unassigned
          226.
          Support SELECT statements in VALUES of INSERT INTO Sub-task Open Unassigned
          227.
          Support [CREATE|DROP] TYPE Sub-task Open Unassigned
          228.
          Support [CREATE|DROP] RULE - define a new plan rewrite rule Sub-task Open Unassigned
          229.
          Support PARTITION BY [RANGE|LIST|HASH] and PARTITION OF in CREATE TABLE Sub-task Open Unassigned
          230.
          Concat for array in Spark SQL is not the one in PostgreSQL but array_cat Sub-task Open Unassigned
          231.
          Date and Timestamp type +/- null should be null as Postgres Sub-task Open Unassigned
          232.
          Support truncate multiple tables Sub-task Open Unassigned

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                smilegator Xiao Li
              • Votes:
                1 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated: