Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14048

REDO operation of WASB#AtomicRename should create placeholder blob for destination folder

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha4
    • fs/azure
    • None
    • Reviewed

    Description

      While doing manual testing, I realized that the crash recovery of AtomicRename operation of a folder in AzureNativeFileSystem doesn't create a placeholder property blob for destination folder. Due to this bug, the destination folder can not be renamed again.

      Below is how I tested this:
      1. Create a test directory as "/test/A"
      2. Create 15 block blobs in "/test/A" folder.
      3. Run "hadoop fs -mv /test/A /test/B" command and crash it as soon as /test/A-RenamePending.json file is created.
      4. Now run "hadoop fs -lsr /test" command, which should complete the pending rename operation (redo) as a part of crash recovery.
      5. The REDO method copies the pending files from source folder to destination folder (by consulting A-RenamePending.json file), but it doesn't create a 0-byte property blob for /test/B folder, which is a bug as that folder will not be usable for many operations.

      Attachments

        1. HADOOP-14048.patch
          2 kB
          NITIN VERMA

        Activity

          People

            nitin.verma@gmail.com NITIN VERMA
            nitin.verma@gmail.com NITIN VERMA
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: