Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14739

add failOnCastException Configuration to Json FormatDescriptor

    XMLWordPrintableJSON

    Details

      Description

      When flink read data from kafka (format is json), the schema is defined, similar to the following DDL

      CREATE TABLE kafka_source (
        intotime VARCHAR,
        userinfo ROW<name VARCHAR,age int>
      ) WITH (
         'connector.type' = 'kafka',
         'format.type' = 'json',
          .............
      )
      
      

      But when flink encounters error data, such as the type of userinfo is a string, the program will throw the following exception and then fail.

      Caused by: java.lang.ClassCastException: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.node.TextNode cannot be cast to org.apache.flink.shaded.jackson2.com.fasterxml.jackson. databind.node.ObjectNode
      

      I want to find the wrong data and don't want the program to fail. So I want to add a json configuration, just like org.apache.flink.table.descriptors.Json#failOnMissingField, which allows the user to configure.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              zhangjun Jun Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: