Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18120

QueryExecutionListener method doesnt' get executed for DataFrameWriter methods

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.1
    • 2.2.0
    • SQL
    • None

    Description

      QueryExecutionListener is a class that has methods named onSuccess() and onFailure() that gets called when a query is executed. Each of those methods takes a QueryExecution object as a parameter which can be used for metrics analysis. It gets called for several of the DataSet methods like take, head, first, collect etc. but doesn't get called for any of the DataFrameWriter methods like saveAsTable, save etc.

      Attachments

        Activity

          People

            cloud_fan Wenchen Fan
            salilsurendran Salil Surendran
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: