Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15461 Improvements over the Hadoop support with Windows
  3. HADOOP-12550

NativeIO#renameTo on Windows cannot replace an existing file at the destination.

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: native
    • Labels:
      None
    • Environment:

      Windows

      Description

      NativeIO#renameTo currently has different semantics on Linux vs. Windows if a file already exists at the destination. On Linux, it's a passthrough to the rename syscall, which will replace an existing file at the destination. On Windows, it's a passthrough to MoveFile, which cannot replace an existing file at the destination and instead triggers an error. The easiest way to observe this difference is to run the HDFS test TestRollingUpgrade#testRollback. This fails on Windows due to a block recovery after truncate trying to replace a block at an existing destination path. This issue proposes to use MoveFileEx on Windows with the MOVEFILE_REPLACE_EXISTING flag.

        Attachments

        1. HADOOP-12550.001.patch
          0.7 kB
          Chris Nauroth
        2. HADOOP-12550.002.patch
          1 kB
          Chris Nauroth

          Issue Links

            Activity

              People

              • Assignee:
                cnauroth Chris Nauroth
                Reporter:
                cnauroth Chris Nauroth
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: