Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7263

Remove obsolete validateTargetPath() from FrameworkUploader

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.1
    • Fix Version/s: 3.3.0
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The following piece of code has been in the FrameworkUploader class for a while:

        private void validateTargetPath() throws UploaderException {
          if (!target.startsWith("hdfs:/") &&
              !target.startsWith("file:/")) {
            throw new UploaderException("Target path is not hdfs or local " + target);
          }
        }
      

      As parts of MR and YARN has evolved, you can run YARN with OzoneFS as storage for instance - so HDFS (or local) is not a requirement anymore. Also it does not make sense to add other suitable filesystems when they're tested against the MR framework, so I'm suggesting to remove the check.

        Attachments

        1. MAPREDUCE-7263.001.patch
          2 kB
          Hudáky Márton Gyula

          Activity

            People

            • Assignee:
              mhudaky Hudáky Márton Gyula
              Reporter:
              adam.antal Adam Antal
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: