Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3813

Support Application timeout feature in YARN.

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0
    • Component/s: scheduler
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      It will be useful to support Application Timeout in YARN. Some use cases are not worried about the output of the applications if the application is not completed in a specific time.

      Background:
      The requirement is to show the CDR statistics of last few minutes, say for every 5 minutes. The same Job will run continuously with different dataset.
      So one job will be started in every 5 minutes. The estimate time for this task is 2 minutes or lesser time.
      If the application is not completing in the given time the output is not useful.

      Proposal
      So idea is to support application timeout, with which timeout parameter is given while submitting the job.
      Here, user is expecting to finish (complete or kill) the application in the given time.

      One option for us is to move this logic to Application client (who submit the job).
      But it will be nice if it can be generic logic and can make more robust.

      Kindly provide your suggestions/opinion on this feature. If it sounds good, i will update the design doc and prototype patch

        Attachments

        1. YARN Application Timeout .pdf
          100 kB
          nijel
        2. 0001-YARN-3813.patch
          31 kB
          nijel
        3. 0002_YARN-3813.patch
          32 kB
          nijel
        4. Yarn Application Timeout.v1.pdf
          301 kB
          Rohith Sharma K S

          Issue Links

            Activity

              People

              • Assignee:
                rohithsharma Rohith Sharma K S
                Reporter:
                nijel nijel
              • Votes:
                1 Vote for this issue
                Watchers:
                30 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: