Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-15585

Improve function identifier string in plan digest

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.11.0
    • Component/s: Table SQL / Planner
    • Labels:
      None

      Description

      Currently, we are using UserDefinedFunction#functionIdentifier as the identifier string of UDFs in plan digest, for example:

      LogicalTableFunctionScan(invocation=[org$apache$flink$table$planner$utils$TableFunc1$8050927803993624f40152a838c98018($2)], rowType=...)
      

      However, the result of UserDefinedFunction#functionIdentifier will change if we just add a method in UserDefinedFunction, because it uses Java serialization. Then we have to update 60 plan tests which is very annoying.

      In the other hand, displaying the function identifier string in operator name in Web UI is verbose to users.

      In order to improve this situation, there are something we can do:
      1) If the UDF has a catalog function name, we can just use the catalog name as the digest. Otherwise, fallback to (2).
      2) If the UDF doesn't contain fields, we just use the full calss name as the digest. Otherwise, fallback to (3).
      3) Use identifier string which will do the full serialization.

        Attachments

          Activity

            People

            • Assignee:
              godfreyhe godfrey he
              Reporter:
              jark Jark Wu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: