[FLINK-11547] JsonMappingException in DynamoDBStreamsSchema

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.0
• Fix Version/s: 1.12.0
• Component/s: Connectors / Kinesis
• Labels: pull-request-available
• Assignee: Devin Thomson
• Reporter: Devin Thomson
• Created: 07/Feb/19 09:24
• Updated: 24/Jul/20 16:28
• Resolved: 24/Jul/20 16:28

Description

If DynamoDBStreamsSchema is used as the deserializer for FlinkDynamoDBStreamsConsumer, an exception occurs during deserialization of a record. The stack trace is attached.

 

This is a blocker for using DynamoDBStreamsSchema, but can be worked around by implementing a custom deserializer. The issue appears to be due to the usage of vanilla ObjectMapper:

 

private static final ObjectMapper MAPPER = new ObjectMapper();

When it should be using the appropriate mix-ins offered by the dynamodb stream adapter library:

private static final ObjectMapper MAPPER = new RecordObjectMapper();

This appears to resolve the issue, I tested by using my own deserializer implementation.

 

Issue Links

• is related to: FLINK-4582 Allow FlinkKinesisConsumer to adapt for AWS DynamoDB Streams
• links to: GitHub Pull Request #12916