Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29766

Aggregate metrics asynchronously in SQL listener

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.4
    • 3.0.0
    • SQL
    • None

    Description

      This is a follow up to SPARK-29562.

      That change made metrics collection faster, and also sped up metrics aggregation. But it is still too slow to execute in an event handler, so we should do it asynchronously to minimize events being dropped by the listener bus.

      Attachments

        Issue Links

          Activity

            People

              vanzin Marcelo Masiero Vanzin
              vanzin Marcelo Masiero Vanzin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: