Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1336 [Umbrella] Work-preserving nodemanager restart
  3. YARN-1338

Recover localized resource cache state upon nodemanager restart

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.5.0
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Today when node manager restarts we clean up all the distributed cache files from disk. This is definitely not ideal from 2 aspects.

      • For work preserving restart we definitely want them as running containers are using them
      • For even non work preserving restart this will be useful in the sense that we don't have to download them again if needed by future tasks.

        Attachments

        1. YARN-1338v6.patch
          165 kB
          Jason Lowe
        2. YARN-1338v5.patch
          157 kB
          Jason Lowe
        3. YARN-1338v4.patch
          156 kB
          Jason Lowe
        4. YARN-1338v3-and-YARN-1987.patch
          163 kB
          Jason Lowe
        5. YARN-1338v2.patch
          156 kB
          Jason Lowe
        6. YARN-1338.patch
          156 kB
          Jason Lowe

          Issue Links

            Activity

              People

              • Assignee:
                jlowe Jason Lowe
                Reporter:
                jlowe Jason Lowe
              • Votes:
                0 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: