Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6810

Add a set of built-in functions to Table API & SQL

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.4.0
    • 1.20.0
    • Table SQL / API
    • None

    Description

      Umbrella issue for adding functions such as mathematical-function LOG, date-functions DATEADD, string-functions LPAD, etc.

      How to contribute a built-in function
      Thank you very much for contributing a built-in function. In order to make sure your contributions are in a good direction, it is recommended to read the following instructions.

      1. Investigate the behavior of the function that you are going to contribute in major DBMSs. This is very important since we have to understand the exact semantics and type inference of the function.
      2. It is recommended to add function for both SQL and Table API (Java, Scala, Python).
      3. For every function, add corresponding docs which should include a SQL, a Java/Scala/Python version in ./docs/dev/table/functions/systemFunctions.md. Make sure your description of the function is accurate. Please do not simply copy documentation from other projects, especially if the projects are not Apache licensed.
      4. Take overflow, handling of NULL, other data types and exceptions into consideration.

      There are different generations of function stacks in the Flink code base. The newest generation has been added in FLINK-20522 and is the recommended way of implementing a function similar to a regular user-defined function.

      Adding a built-in function requires changes in only 2 classes: BuiltInFunctionDefinitions for the definition and the class that contains the actual runtime logic.

      Tests should be added in org.apache.flink.table.planner.functions.

      See the reference implementation of the IFNULL function here:

      https://github.com/apache/flink/commit/df46f8948279d1ed89d7113919b4c4f720cf4199

      Attachments

        1. how to add a scalar function.png
          159 kB
          Xingcan Cui
        1.
        Add TIMESTAMPADD supported in SQL Sub-task Resolved sunjincheng  
        2.
        Add TIMESTAMPADD supported in TableAPI Sub-task Resolved sunjincheng  
        3.
        Add TIMESTAMPDIFF supported in SQL Sub-task Resolved sunjincheng  
        4.
        Add TIMESTAMPDIFF supported in TableAPI Sub-task Resolved Unassigned  
        5.
        Add LOG(X) supported in SQL Sub-task Resolved sunjincheng  
        6.
        ADD LOG(X) supported in TableAPI Sub-task Closed Jiayi Liao  
        7.
        Add L/RPAD supported in SQL Sub-task Resolved sunjincheng  
        8.
        Add L/RPAD supported in TableAPI Sub-task Resolved sunjincheng  
        9.
        Add BIN supported in SQL & Table API Sub-task Resolved sunjincheng  
        10.
        Add BIN supported in TableAPI Sub-task Resolved sunjincheng  
        11.
        Add STR_TO_DATE supported in SQL Sub-task In Progress Unassigned  
        12.
        Add STR_TO_DATE supported in TableAPI Sub-task Open Unassigned  
        13.
        Add CONCAT/CONCAT_WS supported in SQL Sub-task Closed sunjincheng  
        14.
        Add CONCAT/CONCAT_WS supported in TableAPI Sub-task Closed sunjincheng  
        15.
        Add support for MD5, SHA1 and SHA2 Sub-task Closed Michael Gendelman  
        16.
        Add MD5/SHA1/SHA2 supported in TableAPI Sub-task Closed Alexandr Arkhipov  
        17.
        Add E() supported in SQL Sub-task Closed sunjincheng  
        18.
        Add E() supported in TableAPI Sub-task Resolved Timo Walther  
        19.
        Add DATE_FORMAT supported in SQL Sub-task Closed Unassigned  
        20.
        Add UUID supported in TableAPI/SQL Sub-task Resolved Jiayi Liao  
        21.
        Fix Mod scala function bug Sub-task Resolved sunjincheng  
        22.
        Add ASCII/CHR function for Table API Sub-task Closed Unassigned  
        23.
        Add regexp_extract supported in TableAPI and SQL Sub-task Resolved vinoyang  
        24.
        Add regexp_replace supported in TableAPI and SQL Sub-task Resolved vinoyang  
        25.
        Add ISNUMERIC supported in Table API/SQL Sub-task Closed vinoyang  
        26.
        Add LTRIM supported in Table API and SQL Sub-task Resolved vinoyang  
        27.
        Add RTRIM supported in Table API and SQL Sub-task Resolved vinoyang  
        28.
        Add FROM_BASE64 function for table/sql API Sub-task Resolved vinoyang  
        29.
        Add TO_BASE64 function for table/sql API Sub-task Resolved vinoyang  
        30.
        Add REPEAT supported in Table API and SQL Sub-task Resolved vinoyang  
        31.
        Add replace supported in TableAPI and SQL Sub-task Resolved Guibo Pan  
        32.
        Add reverse supported in TableAPI and SQL Sub-task In Progress Unassigned  
        33.
        Add LOG2 function for table/sql API Sub-task Resolved vinoyang  
        34.
        Add REGEXP_MATCH in TableAPI and SQL Sub-task Open Unassigned  
        35.
        Add Cosh math function supported in Table API and SQL Sub-task Closed vinoyang  
        36.
        Add Sinh math function supported in Table API and SQL Sub-task Closed vinoyang  
        37.
        Add Tanh math function supported in Table API and SQL Sub-task Closed vinoyang  
        38.
        Add COTH math function supported in Table API and SQL Sub-task Closed Aleksei Izmalkin
        39.
        Add CBRT math function supported in Table API and SQL Sub-task Closed Aleksei Izmalkin
        40.
        Add HYPOT math function supported in Table API and SQL Sub-task Closed Aleksei Izmalkin
        41.
        Add REMAINDER math function supported in Table API and SQL Sub-task Open Unassigned  
        42.
        Add support for IS DISTINCT FROM in code generator Sub-task Resolved Unassigned  
        43.
        Add support for IS NOT DISTINCT FROM in code generator Sub-task Resolved Unassigned  
        44.
        Add Reverse function supported in Table API and SQL Sub-task Open Unassigned
        45.
        [Bitwise Functions] Add BIT_AND, BIT_OR functions supported in Table API and SQL Sub-task Open Unassigned  
        46.
        [Bitwise Functions] Add BIT_LSHIFT, BIT_RSHIFT functions supported in Table API and SQL Sub-task Open Ran Tao  
        47.
        [Bitwise Functions] Add BIT_XOR function supported in Table API and SQL Sub-task Open Ran Tao  
        48.
        Add contain operator for the array datatype to SQL Sub-task Closed Unassigned  
        49.
        Support GREATEST and LEAST functions in SQL Sub-task Closed Leonard Xu  
        50.
        Support ARRAY_AGG aggregate function Sub-task Closed jiabao.sun  
        51.
        Built-in functions for collections Sub-task Reopened Sergey Nuyanzin  
        52.
        Add support for CURRENT_WATERMARK to SQL Sub-task Closed Ingo Bürk  
        53.
        Add TRANSLATE supported in SQL & Table API Sub-task Open Unassigned  
        54.
        Add SUBSTRING_INDEX supported in SQL & Table API Sub-task Open Unassigned  
        55.
        Add DATE_ADD supported in SQL & Table API Sub-task Open Unassigned  
        56.
        Add ADD_MONTHS supported in SQL and Table API Sub-task Open Unassigned  
        57.
        Add DATE_SUB supported in SQL & Table API Sub-task Open Unassigned  
        58.
        Add SHIFTRIGHT supported in SQL & Table API Sub-task Open Unassigned  
        59.
        Add ARRAY_CONTAINS supported in SQL & Table API Sub-task Closed Timo Walther  
        60.
        Add SORT_ARRAY supported in SQL & Table API Sub-task Closed Hanyu Zheng  
        61.
        Add NVL supported in SQL & Table API Sub-task Open Unassigned  
        62.
        Add HASH supported in SQL & Table API Sub-task Open Unassigned  
        63.
        Add ARRAY_DISTINCT supported in SQL & Table API Sub-task Closed Sergey Nuyanzin  
        64.
        Add ARRAY_APPEND and ARRAY_PREPEND supported in SQL & Table API Sub-task Closed Sergey Nuyanzin  
        65.
        Add ARRAY_REMOVE supported in SQL & Table API Sub-task Closed Jacky Lau  
        66.
        Add ARRAY_SIZE supported in SQL & Table API Sub-task Open Jacky Lau  
        67.
        Add ARRAY_UNION supported in SQL & Table API Sub-task Resolved Jacky Lau  
        68.
        Add MAP_VALUES supported in SQL & Table API Sub-task Closed Unassigned  
        69.
        Add MAP_KEYS supported in SQL & Table API Sub-task Closed Unassigned  
        70.
        Add ARRAY_EXCEPT supported in SQL & Table API Sub-task Closed Hanyu Zheng  
        71.
        Add URL_ENCODE and URL_DECODE function Sub-task Open Unassigned  
        72.
        Add JSON_QUOTE and JSON_UNQUOTE function Sub-task Open Jeyhun Karimov  
        73.
        Support MODE aggregate function Sub-task Open Sergey Nuyanzin  

        Activity

          People

            sunjincheng121 sunjincheng
            sunjincheng121 sunjincheng
            Votes:
            0 Vote for this issue
            Watchers:
            30 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 40m
                40m