Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9430

Support Casting of Object to Primitive types for Flink SQL UDF

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      We want to add a SQL UDF to access specific element in a JSON string using JSON path. However, the JSON element can be of different types, e.g. Int, Float, Double, String, Boolean and etc.. Since return type is not part of the method signature, we can not use overload. So we will end up writing a UDF for each type, e.g. GetFloatFromJSON, GetIntFromJSON and etc., which has a lot of duplication. 

      One way to unify all these UDF functions is to implement one UDF and return java.lang.Object, and in the SQL statement, use CAST AS to cast the returned Object into the correct type. Below is an example:

       

      object JsonPathUDF extends ScalarFunction {
       def eval(jsonStr: String, path: String): Object = {
         JSONParser.parse(jsonStr).read(path)
       }
      }
       SELECT CAST(jsonpath(json, "$.store.book.title") AS VARCHAR(32)) as bookTitle FROM table1

      The current Flink SQL cast implementation does not support casting from GenericTypeInfo<java.lang.Object> to another type, I have already got a local branch to fix this. Please comment if there are alternatives to the problem above.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            suez1224 Shuyi Chen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment