[FLINK-8854] Mapping of SchemaValidator.deriveFieldMapping() is incorrect.

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.5.0
• Component/s: Table SQL / API
• Assignee: Timo Walther
• Reporter: Fabian Hueske
• Created: 04/Mar/18 18:51
• Updated: 15/Mar/18 16:49
• Resolved: 09/Mar/18 13:02

Description

The field mapping returned by SchemaValidator.deriveFieldMapping() is not correct.

It should not only include all fields of the table schema, but also all fields of the format schema (mapped to themselves). Otherwise, it is not possible to use a timestamp extractor on a field that is not in table schema.

For example this configuration would fail:

sources:
  - name: TaxiRides
    schema:
      - name: rideId
        type: LONG
      - name: rowTime
        type: TIMESTAMP
        rowtime:
          timestamps:
            type: "from-field"
            from: "rideTime"
          watermarks:
            type: "periodic-bounded"
            delay: "60000"
    connector:
      ....
    format:
      property-version: 1
      type: json
      schema: "ROW(rideId LONG, rideTime TIMESTAMP)"

because rideTime is not in the table schema.

Issue Links

• links to: GitHub Pull Request #5662