Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1522

FileInputFormat may change the file system of an input path

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 0.20.3
    • None
    • None
    • Reviewed

    Description

      org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(Job job, Path path) uses the default FileSystem but not the FileSystem specified in the path.

      //org.apache.hadoop.mapreduce.lib.input.FileInputFormat
        public static void addInputPath(Job job, 
                                        Path path) throws IOException {
          Configuration conf = job.getConfiguration();
          FileSystem fs = FileSystem.get(conf);
          path = path.makeQualified(fs); // the original FileSystem is lost.
          ...
        }
      

      There is a similar problem in FileInputFormat.setInputPaths(..).

      Attachments

        1. m1522_20100224_0.20.patch
          4 kB
          Tsz-wo Sze
        2. m1522_20100224.patch
          4 kB
          Tsz-wo Sze
        3. m1522_20100301_0.20.patch
          4 kB
          Tsz-wo Sze
        4. m1522_20100301_0.21.patch
          5 kB
          Tsz-wo Sze
        5. m1522_20100301.patch
          4 kB
          Tsz-wo Sze
        6. M1522-1.patch
          4 kB
          Christopher Douglas
        7. M1522-1v20.patch
          4 kB
          Christopher Douglas
        8. M1522-1v21.patch
          4 kB
          Christopher Douglas
        9. m1552_20100223_0.20.patch
          2 kB
          Tsz-wo Sze

        Issue Links

          Activity

            People

              szetszwo Tsz-wo Sze
              szetszwo Tsz-wo Sze
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: