[FLINK-8162] Kinesis Connector to report millisBehindLatest metric

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: Connectors / Kinesis
• Labels: kinesis
• Assignee: Unassigned
• Reporter: Cristian
• Created: 27/Nov/17 23:00
• Updated: 13/Jan/18 08:11
• Resolved: 13/Jan/18 08:11

Description

When reading from Kinesis streams, one of the most valuable metrics is "MillisBehindLatest" (see https://github.com/aws/aws-sdk-java/blob/25f0821f69bf94ec456f602f2b83ea2b0ca15643/aws-java-sdk-kinesis/src/main/java/com/amazonaws/services/kinesis/model/GetRecordsResult.java#L187-L201).

Flink should use its metrics mechanism to report this value as a gauge, tagging it with the shard id.

Issue Links

• is duplicated by: FLINK-8219 Kinesis Connector metrics
• links to: GitHub Pull Request #5182