Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24637

Add metrics regarding state and watermark to dropwizard metrics

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.0
    • Component/s: Structured Streaming
    • Labels:
      None

      Description

      Though Spark provides option to enable stream metrics into Dropwizard, it only exposes three metrics: "inputRate-total", "processingRate-total", "latency" which are not enough information to operate.

      Since Spark already exposes other metrics as well which are valuable for operation perspective, we could pick some and expose them into Dropwizard as well.

        Attachments

          Activity

            People

            • Assignee:
              kabhwan Jungtaek Lim
              Reporter:
              kabhwan Jungtaek Lim
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: