Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16839

Connectors Kinesis metrics can be disabled

    XMLWordPrintableJSON

    Details

      Description

      Currently there are 9 metrics in the kinesis connector, each of which is recorded according to the kinesis shard dimension. If there are enough shards, taskmanager mtrics will be unavailable.

      In our production environment, a case of a job that reads dynamodb stream kinesis adapter, this dynamodb has more than 10,000 shards, multiplied by 9 metrics, there are more than 100,000 metrics for kinesis, and the entire metrics output reaches tens of MB , Cause prometheus cannot collect metrics.

      We should have a configuration item that can disable kinesis metrics

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              muchl Muchl
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: