Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-8854

Mapping of SchemaValidator.deriveFieldMapping() is incorrect.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.5.0
    • Component/s: Table API & SQL
    • Labels:
      None

      Description

      The field mapping returned by SchemaValidator.deriveFieldMapping() is not correct.

      It should not only include all fields of the table schema, but also all fields of the format schema (mapped to themselves). Otherwise, it is not possible to use a timestamp extractor on a field that is not in table schema.

      For example this configuration would fail:

      sources:
        - name: TaxiRides
          schema:
            - name: rideId
              type: LONG
            - name: rowTime
              type: TIMESTAMP
              rowtime:
                timestamps:
                  type: "from-field"
                  from: "rideTime"
                watermarks:
                  type: "periodic-bounded"
                  delay: "60000"
          connector:
            ....
          format:
            property-version: 1
            type: json
            schema: "ROW(rideId LONG, rideTime TIMESTAMP)"
      

      because rideTime is not in the table schema.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                twalthr Timo Walther
                Reporter:
                fhueske Fabian Hueske
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: