Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14049

Add functionality in spark history sever API to query applications by end time

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.6.1, 2.0.0
    • 2.2.0
    • Spark Core
    • None

    Description

      Currently, spark history server REST API provides functionality to query applications by application start time range based on minDate and maxDate query parameters, but it lacks support to query applications by their end time. In this Jira we are proposing optional minEndDate and maxEndDate query parameters and filtering capability based on these parameters to spark history server REST API. This functionality can be used for following queries,

      1. Applications finished in last 'x' minutes
      2. Applications finished before 'y' time
      3. Applications finished between 'x' time to 'y' time
      4. Applications started from 'x' time and finished before 'y' time.

      For backward compatibility, we can keep existing minDate and maxDate query parameters as they are and they can continue support filtering based on start time range.

      Attachments

        Activity

          People

            paragpc Parag Chaudhari
            paragpc Parag Chaudhari
            Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: