Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1522

FileInputFormat may change the file system of an input path

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.3
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(Job job, Path path) uses the default FileSystem but not the FileSystem specified in the path.

      //org.apache.hadoop.mapreduce.lib.input.FileInputFormat
        public static void addInputPath(Job job, 
                                        Path path) throws IOException {
          Configuration conf = job.getConfiguration();
          FileSystem fs = FileSystem.get(conf);
          path = path.makeQualified(fs); // the original FileSystem is lost.
          ...
        }
      

      There is a similar problem in FileInputFormat.setInputPaths(..).

        Attachments

        1. m1522_20100224_0.20.patch
          4 kB
          Tsz Wo Nicholas Sze
        2. m1522_20100224.patch
          4 kB
          Tsz Wo Nicholas Sze
        3. m1522_20100301_0.20.patch
          4 kB
          Tsz Wo Nicholas Sze
        4. m1522_20100301_0.21.patch
          5 kB
          Tsz Wo Nicholas Sze
        5. m1522_20100301.patch
          4 kB
          Tsz Wo Nicholas Sze
        6. M1522-1.patch
          4 kB
          Chris Douglas
        7. M1522-1v20.patch
          4 kB
          Chris Douglas
        8. M1522-1v21.patch
          4 kB
          Chris Douglas
        9. m1552_20100223_0.20.patch
          2 kB
          Tsz Wo Nicholas Sze

          Issue Links

            Activity

              People

              • Assignee:
                szetszwo Tsz Wo Nicholas Sze
                Reporter:
                szetszwo Tsz Wo Nicholas Sze
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: