Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15461 Improvements over the Hadoop support with Windows
  3. HADOOP-12550

NativeIO#renameTo on Windows cannot replace an existing file at the destination.

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • native
    • None
    • Windows

    Description

      NativeIO#renameTo currently has different semantics on Linux vs. Windows if a file already exists at the destination. On Linux, it's a passthrough to the rename syscall, which will replace an existing file at the destination. On Windows, it's a passthrough to MoveFile, which cannot replace an existing file at the destination and instead triggers an error. The easiest way to observe this difference is to run the HDFS test TestRollingUpgrade#testRollback. This fails on Windows due to a block recovery after truncate trying to replace a block at an existing destination path. This issue proposes to use MoveFileEx on Windows with the MOVEFILE_REPLACE_EXISTING flag.

      Attachments

        1. HADOOP-12550.002.patch
          1 kB
          Chris Nauroth
        2. HADOOP-12550.001.patch
          0.7 kB
          Chris Nauroth

        Issue Links

          Activity

            People

              cnauroth Chris Nauroth
              cnauroth Chris Nauroth
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: