Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2005

Blacklisting support for scheduling AMs

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.23.10, 2.4.0
    • 2.8.0, 3.0.0-alpha1
    • resourcemanager
    • None
    • Reviewed

    Description

      It would be nice if the RM supported blacklisting a node for an AM launch after the same node fails a configurable number of AM attempts. This would be similar to the blacklisting support for scheduling task attempts in the MapReduce AM but for scheduling AM attempts on the RM side.

      Attachments

        1. YARN-2005.001.patch
          40 kB
          Anubhav Dhoot
        2. YARN-2005.002.patch
          40 kB
          Anubhav Dhoot
        3. YARN-2005.003.patch
          41 kB
          Anubhav Dhoot
        4. YARN-2005.004.patch
          46 kB
          Anubhav Dhoot
        5. YARN-2005.005.patch
          48 kB
          Anubhav Dhoot
        6. YARN-2005.006.patch
          54 kB
          Anubhav Dhoot
        7. YARN-2005.006.patch
          54 kB
          Anubhav Dhoot
        8. YARN-2005.007.patch
          63 kB
          Anubhav Dhoot
        9. YARN-2005.008.patch
          67 kB
          Anubhav Dhoot
        10. YARN-2005.009.patch
          64 kB
          Anubhav Dhoot

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-6511 MRAppMaster second attempt starting on the same node as a previously failed MRAppMaster attempt

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-2293 Scoring for NMs to identify a better candidate to launch AMs

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-3744 ResourceManager should avoid allocating AM to same node repeatedly in case of AM launch failures

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. YARN-896 Roll up for long-lived services in YARN

          • Major - Major loss of function.
          • Open
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. YARN-4217 Failed AM attempt retries on same failed host

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-8352 AM should retry on a different node after the previous application attempt fail

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. YARN-3994 RM should respect AM resource/placement constraints

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. YARN-4837 User facing aspects of 'AM blacklisting' feature need fixing

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-3803 Application hangs after more then one localization attempt fails on the same NM

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-4576 Enhancement for tracking Blacklist in AM Launching

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-1073 NM to recognise when it can't spawn process and stop accepting containers

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. YARN-4247 Deadlock in FSAppAttempt and RMAppAttemptImpl causes RM to stop processing events

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-4685 Disable AM blacklisting by default to mitigate situations that application get hanged

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-4284 condition for AM blacklisting is too narrow

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-4670 add logging when a node is AM-blacklisted

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-964 Give a parameter that can set AM retry interval

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              adhoot Anubhav Dhoot
              jlowe Jason Darrell Lowe
              Votes:
              3 Vote for this issue
              Watchers:
              38 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: