Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14739

add failOnCastException Configuration to Json FormatDescriptor

    XMLWordPrintableJSON

Details

    Description

      When flink read data from kafka (format is json), the schema is defined, similar to the following DDL

      CREATE TABLE kafka_source (
        intotime VARCHAR,
        userinfo ROW<name VARCHAR,age int>
      ) WITH (
         'connector.type' = 'kafka',
         'format.type' = 'json',
          .............
      )
      
      

      But when flink encounters error data, such as the type of userinfo is a string, the program will throw the following exception and then fail.

      Caused by: java.lang.ClassCastException: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.node.TextNode cannot be cast to org.apache.flink.shaded.jackson2.com.fasterxml.jackson. databind.node.ObjectNode
      

      I want to find the wrong data and don't want the program to fail. So I want to add a json configuration, just like org.apache.flink.table.descriptors.Json#failOnMissingField, which allows the user to configure.

      Attachments

        Activity

          People

            Unassigned Unassigned
            zhangjun Jun Zhang
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: