Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9590

Move to JDK7 improved APIs for file operations when available

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      JDK6 does not have a complete support for local file system file operations. Specifically:

      • There is no symlink/hardlink APIs what forced Hadoop to defer to shell based tooling
      • No error information returned when File#mkdir/mkdirs or File#renameTo fails, making it unnecessary hard to troubleshoot some issues
      • File#canRead/canWrite/canExecute do not perform any access checks on Windows making APIs inconsistent with the Unix behavior
      • File#setReadable/setWritable/setExecutable do not change access rights on Windows making APIs inconsistent with the Unix behavior
      • File#length does not work as expected on symlinks on Windows
      • File#renameTo does not work as expected on symlinks on Windows

      All above resulted in Hadoop community having to fill in the gaps by providing equivalent native implementations or applying workarounds.

      JDK7 addressed (as far as I know) all (or most) of the above problems, either thru the newly introduced Files class or thru bug fixes.

      This is a tracking Jira to revisit above mediations once JDK7 becomes the supported platform by the Hadoop community. This work would allow significant portion of the native platform-dependent code to be replaced with Java equivalents what is goodness w.r.t. Hadoop cross-platform support.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ivanmi Ivan Mitic
              • Votes:
                1 Vote for this issue
                Watchers:
                20 Start watching this issue

                Dates

                • Created:
                  Updated: