Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1522

FileInputFormat may change the file system of an input path

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.3
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(Job job, Path path) uses the default FileSystem but not the FileSystem specified in the path.

      //org.apache.hadoop.mapreduce.lib.input.FileInputFormat
        public static void addInputPath(Job job, 
                                        Path path) throws IOException {
          Configuration conf = job.getConfiguration();
          FileSystem fs = FileSystem.get(conf);
          path = path.makeQualified(fs); // the original FileSystem is lost.
          ...
        }
      

      There is a similar problem in FileInputFormat.setInputPaths(..).

      1. m1522_20100224_0.20.patch
        4 kB
        Tsz Wo Nicholas Sze
      2. m1522_20100224.patch
        4 kB
        Tsz Wo Nicholas Sze
      3. m1522_20100301_0.20.patch
        4 kB
        Tsz Wo Nicholas Sze
      4. m1522_20100301_0.21.patch
        5 kB
        Tsz Wo Nicholas Sze
      5. m1522_20100301.patch
        4 kB
        Tsz Wo Nicholas Sze
      6. M1522-1.patch
        4 kB
        Chris Douglas
      7. M1522-1v20.patch
        4 kB
        Chris Douglas
      8. M1522-1v21.patch
        4 kB
        Chris Douglas
      9. m1552_20100223_0.20.patch
        2 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Gavin made changes -
          Link This issue is depended upon by PIG-1378 [ PIG-1378 ]
          Gavin made changes -
          Link This issue blocks PIG-1378 [ PIG-1378 ]
          Viraj Bhat made changes -
          Link This issue blocks PIG-1378 [ PIG-1378 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Chris Douglas made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Chris Douglas made changes -
          Attachment M1522-1.patch [ 12437996 ]
          Chris Douglas made changes -
          Attachment M1522-1v21.patch [ 12437995 ]
          Chris Douglas made changes -
          Attachment M1522-1v20.patch [ 12437994 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment m1522_20100301_0.21.patch [ 12437863 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment m1522_20100301_0.20.patch [ 12437806 ]
          Chris Douglas made changes -
          Fix Version/s 0.20.3 [ 12314813 ]
          Priority Major [ 3 ] Blocker [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee Tsz Wo (Nicholas), SZE [ szetszwo ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment m1522_20100301.patch [ 12437503 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment m1522_20100224.patch [ 12437502 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment m1522_20100224.patch [ 12437502 ]
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment m1522_20100224.patch [ 12436958 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment m1522_20100224_0.20.patch [ 12436957 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment m1552_20100223_0.20.patch [ 12436825 ]
          Tsz Wo Nicholas Sze made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-6564 [ HADOOP-6564 ]
          Tsz Wo Nicholas Sze created issue -

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development