Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4150 Versioning and rolling upgrades for MR2
  3. MAPREDUCE-5891

Improved shuffle error handling across NM restarts

          Details

          • Type: Sub-task
          • Status: Closed
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: 2.5.0
          • Fix Version/s: 2.6.0
          • Component/s: None
          • Labels:
            None
          • Target Version/s:
          • Hadoop Flags:
            Reviewed

            Description

            To minimize the number of map fetch failures reported by reducers across an NM restart it would be nice if reducers only reported a fetch failure after trying for at specified period of time to retrieve the data.

              Attachments

              1. MAPREDUCE-5891-v6.patch
                28 kB
                Junping Du
              2. MAPREDUCE-5891-v5.patch
                28 kB
                Junping Du
              3. MAPREDUCE-5891-v4.patch
                28 kB
                Junping Du
              4. MAPREDUCE-5891-v3.patch
                28 kB
                Junping Du
              5. MAPREDUCE-5891-v2.patch
                27 kB
                Junping Du
              6. MAPREDUCE-5891-demo.patch
                14 kB
                Junping Du
              7. MAPREDUCE-5891.patch
                26 kB
                Junping Du

                Issue Links

                breaks

                Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-6303 Read timeout when retrying a fetch error can be fatal to a reducer

                • Blocker - Blocks development and/or testing work, production could not run
                • Closed
                is related to

                Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-6156 Fetcher - connect() doesn't handle connection refused correctly

                • Blocker - Blocks development and/or testing work, production could not run
                • Closed

                Improvement - An improvement or enhancement to an existing feature or task. YARN-666 [Umbrella] Support rolling upgrades in YARN

                • Major - Major loss of function.
                • Closed
                relates to

                New Feature - A new feature of the product, which has yet to be developed. YARN-1336 [Umbrella] Work-preserving nodemanager restart

                • Major - Major loss of function.
                • Resolved

                  Activity

                    People

                    • Assignee:
                      djp Junping Du
                      Reporter:
                      jlowe Jason Lowe
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      11 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: