Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: HDFS-6581
    • Fix Version/s: 2.6.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      It used to be very unlikely that the directory scanner encountered two replicas of the same block on different volumes.

      With memory storage, it is very likely to hit this with the following sequence of events:

      1. Block is written to RAM disk
      2. Lazy writer saves a copy on persistent volume
      3. DN attempts to evict the original replica from RAM disk, file deletion fails as the replica is in use.
      4. Directory scanner finds a replica on both RAM disk and persistent storage.

      The directory scanner should never delete the block on persistent storage.

        Attachments

        1. HDFS-6978.01.patch
          23 kB
          Arpit Agarwal
        2. HDFS-6978.02.patch
          25 kB
          Arpit Agarwal

          Activity

            People

            • Assignee:
              arpitagarwal Arpit Agarwal
              Reporter:
              arpitagarwal Arpit Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: