Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6608 Work Preserving AM Restart for MapReduce
  3. MAPREDUCE-6726

YARN Registry based AM discovery with retry and in-flight task persistent via JHS



    • Type: Sub-task
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.2.2
    • Component/s: applicationmaster
    • Labels:


      Several tasks will be achieved in this JIRA based on the demo patch in MAPREDUCE-6608:
      1. AM discovery base on YARN register service. Could be replaced by YARN-4758 later due to scale up issue.
      2. Retry logic for TaskUmbilicalProtocol RPC connection
      3. In-flight task recover after AM restart via JHS
      4. Configuration to control the behavior compatible with previous when not enable this feature (by default).
      All security related issues and other concerns discussed in MAPREDUCE-6608 will be addressed in follow up JIRAs.


        1. MAPREDUCE-6726-MAPREDUCE-6608.001.patch
          131 kB
          Srikanth Sampath
        2. WorkPreservingMRAppMaster.pdf
          112 kB
          Srikanth Sampath
        3. MAPREDUCE-6726-MAPREDUCE-6608.001.patch
          131 kB
          Junping Du
        4. MAPREDUCE-6726-MAPREDUCE-6608.002.patch
          121 kB
          Srikanth Sampath
        5. MAPREDUCE-6726-MAPREDUCE-6608.003.patch
          125 kB
          Srikanth Sampath
        6. MAPREDUCE-6726.004.patch
          229 kB
          D M Murali Krishna Reddy
        7. TestReport.pdf
          30 kB
          D M Murali Krishna Reddy
        8. MAPREDUCE-6726.005.trunk.patch
          229 kB
          D M Murali Krishna Reddy



            • Assignee:
              dmmkr D M Murali Krishna Reddy
              junping_du Junping Du
            • Votes:
              0 Vote for this issue
              12 Start watching this issue


              • Created: