Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1967

Add a monitoring API on DAGClient which returns after a time interval or on DAG state change

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.7.0
    • 0.7.0
    • None
    • None

    Description

      To monitor a running DAG, clients end up using DAGClient.getDAGSstatus in a loop with a poll interval.
      In the worst case, they find out about DAG completion, failure etc only after the poll interval.

      Instead, an API can be added which waits on the AM for a specified interval, but can return earlier if the DAG state changes.

      This will end up blocking RPC handlers - but that isn't a problem since we don't have many entities querying for DAG status.

      Attachments

        1. TEZ-1967.1.patch
          9 kB
          Vasanth kumar RJ
        2. TEZ-1967-InitialReview.patch
          17 kB
          Vasanth kumar RJ
        3. TEZ-1967.2.patch
          17 kB
          Vasanth kumar RJ
        4. TEZ-1967.3.patch
          17 kB
          Vasanth kumar RJ
        5. TEZ-1967.4.patch
          20 kB
          Vasanth kumar RJ
        6. TEZ-1967.5.patch
          20 kB
          Vasanth kumar RJ

        Activity

          People

            vasanthkumar Vasanth kumar RJ
            sseth Siddharth Seth
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: