Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5177

Make Node-Manager Download-Resource Component extensible.

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 2.7.0
    • None
    • nodemanager

    Description

      Problem:

      • Downloading files to a local machine/node is called "resource-localization".
      • There are two components that perform resource-location (PublicLocalizer and ComponetLocalizers)
      • Both components utilizes FSDownload.class to perform their downloads.
      • We need a custom implementation of FSDownload.

      Solution:

      • With this change, we make FSDownload.class extensible by wrapping it in a new ResourceDownloader.interface
      • We also update the PublicLocalizer and ComponetLocalizers to load ResourceDownloader rather than FSDownload.
      • NOTE: We use reflection to load the right implementation during runtime.

      Attachments

        1. YARN-5177-V0.patch
          16 kB
          Emeka-NMC

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            nnaemeka Emeka-NMC
            nnaemeka Emeka-NMC

            Dates

              Created:
              Updated:

              Slack

                Issue deployment