Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5177

Make Node-Manager Download-Resource Component extensible.

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.7.0
    • Fix Version/s: None
    • Component/s: nodemanager
    • Labels:
    • Target Version/s:

      Description

      Problem:

      • Downloading files to a local machine/node is called "resource-localization".
      • There are two components that perform resource-location (PublicLocalizer and ComponetLocalizers)
      • Both components utilizes FSDownload.class to perform their downloads.
      • We need a custom implementation of FSDownload.

      Solution:

      • With this change, we make FSDownload.class extensible by wrapping it in a new ResourceDownloader.interface
      • We also update the PublicLocalizer and ComponetLocalizers to load ResourceDownloader rather than FSDownload.
      • NOTE: We use reflection to load the right implementation during runtime.

        Attachments

        1. YARN-5177-V0.patch
          16 kB
          Emeka-NMC

          Activity

            People

            • Assignee:
              nnaemeka Emeka-NMC
              Reporter:
              nnaemeka Emeka-NMC
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: