Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1336 [Umbrella] Work-preserving nodemanager restart
  3. YARN-1338

Recover localized resource cache state upon nodemanager restart

          Details

          • Type: Sub-task
          • Status: Closed
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: 2.3.0
          • Fix Version/s: 2.5.0
          • Component/s: nodemanager
          • Labels:
            None
          • Target Version/s:
          • Hadoop Flags:
            Reviewed

            Description

            Today when node manager restarts we clean up all the distributed cache files from disk. This is definitely not ideal from 2 aspects.

            • For work preserving restart we definitely want them as running containers are using them
            • For even non work preserving restart this will be useful in the sense that we don't have to download them again if needed by future tasks.

              Attachments

              1. YARN-1338.patch
                156 kB
                Jason Lowe
              2. YARN-1338v2.patch
                156 kB
                Jason Lowe
              3. YARN-1338v3-and-YARN-1987.patch
                163 kB
                Jason Lowe
              4. YARN-1338v4.patch
                156 kB
                Jason Lowe
              5. YARN-1338v5.patch
                157 kB
                Jason Lowe
              6. YARN-1338v6.patch
                165 kB
                Jason Lowe

                Issue Links

                blocks

                Sub-task - The sub-task of the issue YARN-1337 Recover containers upon nodemanager restart

                • Major - Major loss of function.
                • Closed
                requires

                Improvement - An improvement or enhancement to an existing feature or task. YARN-1987 Wrapper for leveldb DBIterator to aid in handling database exceptions

                • Major - Major loss of function.
                • Closed

                  Activity

                    People

                    • Assignee:
                      jlowe Jason Lowe
                      Reporter:
                      jlowe Jason Lowe
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      14 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: