Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14048

REDO operation of WASB#AtomicRename should create placeholder blob for destination folder

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha4
    • Component/s: fs/azure
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      While doing manual testing, I realized that the crash recovery of AtomicRename operation of a folder in AzureNativeFileSystem doesn't create a placeholder property blob for destination folder. Due to this bug, the destination folder can not be renamed again.

      Below is how I tested this:
      1. Create a test directory as "/test/A"
      2. Create 15 block blobs in "/test/A" folder.
      3. Run "hadoop fs -mv /test/A /test/B" command and crash it as soon as /test/A-RenamePending.json file is created.
      4. Now run "hadoop fs -lsr /test" command, which should complete the pending rename operation (redo) as a part of crash recovery.
      5. The REDO method copies the pending files from source folder to destination folder (by consulting A-RenamePending.json file), but it doesn't create a 0-byte property blob for /test/B folder, which is a bug as that folder will not be usable for many operations.

        Attachments

          Activity

            People

            • Assignee:
              nitin.verma@gmail.com NITIN VERMA
              Reporter:
              nitin.verma@gmail.com NITIN VERMA
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved: