Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5951

Add support for the YARN Shared Cache

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0
    • None
    • Hide
      MapReduce support for the YARN shared cache allows MapReduce jobs to take advantage of additional resource caching. This saves network bandwidth between the job submission client as well as within the YARN cluster itself. This will reduce job submission time and overall job runtime.
      Show
      MapReduce support for the YARN shared cache allows MapReduce jobs to take advantage of additional resource caching. This saves network bandwidth between the job submission client as well as within the YARN cluster itself. This will reduce job submission time and overall job runtime.

    Description

      Implement the necessary changes so that the MapReduce application can leverage the new YARN shared cache (i.e. YARN-1492).

      Specifically, allow per-job configuration so that MapReduce jobs can specify which set of resources they would like to cache (i.e. jobjar, libjars, archives, files).

      Attachments

        1. MAPREDUCE-5951-trunk-v1.patch
          81 kB
          Chris Trezzo
        2. MAPREDUCE-5951-trunk-v2.patch
          84 kB
          Chris Trezzo
        3. MAPREDUCE-5951-trunk-v3.patch
          89 kB
          Chris Trezzo
        4. MAPREDUCE-5951-trunk-v4.patch
          89 kB
          Chris Trezzo
        5. MAPREDUCE-5951-trunk-v5.patch
          90 kB
          Chris Trezzo
        6. MAPREDUCE-5951-trunk-v6.patch
          103 kB
          Chris Trezzo
        7. MAPREDUCE-5951-trunk-v7.patch
          83 kB
          Chris Trezzo
        8. MAPREDUCE-5951-trunk-v8.patch
          86 kB
          Chris Trezzo
        9. MAPREDUCE-5951-trunk-v9.patch
          86 kB
          Chris Trezzo
        10. MAPREDUCE-5951-trunk-v10.patch
          83 kB
          Chris Trezzo
        11. MAPREDUCE-5951-trunk-v11.patch
          74 kB
          Chris Trezzo
        12. MAPREDUCE-5951-trunk-v12.patch
          72 kB
          Chris Trezzo
        13. MAPREDUCE-5951-trunk-v13.patch
          72 kB
          Chris Trezzo
        14. MAPREDUCE-5951-trunk-v14.patch
          75 kB
          Chris Trezzo
        15. MAPREDUCE-5951-trunk-v15.patch
          75 kB
          Chris Trezzo
        16. MAPREDUCE-5951-trunk.016.patch
          101 kB
          Chris Trezzo
        17. MAPREDUCE-5951-trunk.017.patch
          76 kB
          Chris Trezzo
        18. MAPREDUCE-5951-trunk.018.patch
          76 kB
          Chris Trezzo
        19. MAPREDUCE-5951-Overview.001.pdf
          68 kB
          Chris Trezzo
        20. MAPREDUCE-5951-trunk.019.patch
          78 kB
          Chris Trezzo
        21. MAPREDUCE-5951-trunk-020.patch
          90 kB
          Chris Trezzo
        22. MAPREDUCE-5951-trunk-021.patch
          109 kB
          Chris Trezzo

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ctrezzo Chris Trezzo
            ctrezzo Chris Trezzo
            Votes:
            0 Vote for this issue
            Watchers:
            20 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment