Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5951

Add support for the YARN Shared Cache

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0
    • None
    • Hide
      MapReduce support for the YARN shared cache allows MapReduce jobs to take advantage of additional resource caching. This saves network bandwidth between the job submission client as well as within the YARN cluster itself. This will reduce job submission time and overall job runtime.
      Show
      MapReduce support for the YARN shared cache allows MapReduce jobs to take advantage of additional resource caching. This saves network bandwidth between the job submission client as well as within the YARN cluster itself. This will reduce job submission time and overall job runtime.

    Description

      Implement the necessary changes so that the MapReduce application can leverage the new YARN shared cache (i.e. YARN-1492).

      Specifically, allow per-job configuration so that MapReduce jobs can specify which set of resources they would like to cache (i.e. jobjar, libjars, archives, files).

      Attachments

        1. MAPREDUCE-5951-trunk-021.patch
          109 kB
          Chris Trezzo
        2. MAPREDUCE-5951-trunk-020.patch
          90 kB
          Chris Trezzo
        3. MAPREDUCE-5951-trunk.019.patch
          78 kB
          Chris Trezzo
        4. MAPREDUCE-5951-Overview.001.pdf
          68 kB
          Chris Trezzo
        5. MAPREDUCE-5951-trunk.018.patch
          76 kB
          Chris Trezzo
        6. MAPREDUCE-5951-trunk.017.patch
          76 kB
          Chris Trezzo
        7. MAPREDUCE-5951-trunk.016.patch
          101 kB
          Chris Trezzo
        8. MAPREDUCE-5951-trunk-v15.patch
          75 kB
          Chris Trezzo
        9. MAPREDUCE-5951-trunk-v14.patch
          75 kB
          Chris Trezzo
        10. MAPREDUCE-5951-trunk-v13.patch
          72 kB
          Chris Trezzo
        11. MAPREDUCE-5951-trunk-v12.patch
          72 kB
          Chris Trezzo
        12. MAPREDUCE-5951-trunk-v11.patch
          74 kB
          Chris Trezzo
        13. MAPREDUCE-5951-trunk-v10.patch
          83 kB
          Chris Trezzo
        14. MAPREDUCE-5951-trunk-v9.patch
          86 kB
          Chris Trezzo
        15. MAPREDUCE-5951-trunk-v8.patch
          86 kB
          Chris Trezzo
        16. MAPREDUCE-5951-trunk-v7.patch
          83 kB
          Chris Trezzo
        17. MAPREDUCE-5951-trunk-v6.patch
          103 kB
          Chris Trezzo
        18. MAPREDUCE-5951-trunk-v5.patch
          90 kB
          Chris Trezzo
        19. MAPREDUCE-5951-trunk-v4.patch
          89 kB
          Chris Trezzo
        20. MAPREDUCE-5951-trunk-v3.patch
          89 kB
          Chris Trezzo
        21. MAPREDUCE-5951-trunk-v2.patch
          84 kB
          Chris Trezzo
        22. MAPREDUCE-5951-trunk-v1.patch
          81 kB
          Chris Trezzo

        Issue Links

          Activity

            People

              ctrezzo Chris Trezzo
              ctrezzo Chris Trezzo
              Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: