Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-125

LocalFileSystem.makeAbsolute bug on Windows

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1.1, 0.2.0
    • Component/s: fs
    • Labels:
      None
    • Environment:

      Windows

      Description

      LocalFileSystem.makeAbsolute() has a bug when running on Windows (which is very useful for the development phase of a Hadoop task on one's laptop).

      Problem: if a pathname such as /tmp/hadoop... is given in a config file, when the jobconf file is created, it is put into the relative directory named: currentdir/tmp/hadoop..., but when hadoop tries to open the file, it looks in c:/tmp/hadoop..., and the job fails.

      Cause: while Unix has two kinds of filespecs (relative and absolute), WIndows actually has three:

      (1) relative to current directory (subdir/file)
      (2) relative to current disk (/dir/subdir/file)
      (3) absolute (c:/dir/subdir/file)

      So when a config file specifies a directory with what-is-on-unix an absolute path (/tmp/hadoop...), the makeAbsolute() method will not work correctly. Basically, File.isAbsolute() will return false for cases (1) and (2) above, but true for case (3), which is not expected by the code below.

      The solution would be to code explicit detection of all three casses for Windows in the code below from LocalFileSystem:

      private File makeAbsolute(File f) {
      if (f.isAbsolute())

      { return f; }

      else

      { return new File(workingDir, f.toString()); }

      }

      Im happy to explain if this explanation is confusing...

        Attachments

          Activity

            People

            • Assignee:
              cutting Doug Cutting
              Reporter:
              psutter Peter Sutter
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: