Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7960

The full block report should prune zombie storages even if they're not empty

    XMLWordPrintableJSON

Details

    Description

      The full block report should prune zombie storages even if they're not empty. We have seen cases in production where zombie storages have not been pruned subsequent to HDFS-7575. This could arise any time the NameNode thinks there is a block in some old storage which is actually not there. In this case, the block will not show up in the "new" storage (once old is renamed to new) and the old storage will linger forever as a zombie, even with the HDFS-7596 fix applied. This also happens with datanode hotplug, when a drive is removed. In this case, an entire storage (volume) goes away but the blocks do not show up in another storage on the same datanode.

      Attachments

        1. HDFS-7960-branch-2.6.1.txt
          55 kB
          Vinod Kumar Vavilapalli
        2. HDFS-7960.008.patch
          56 kB
          Lei (Eddy) Xu
        3. HDFS-7960.007.patch
          54 kB
          Lei (Eddy) Xu
        4. HDFS-7960.006.patch
          56 kB
          Colin McCabe
        5. HDFS-7960.005.patch
          56 kB
          Colin McCabe
        6. HDFS-7960.004.patch
          46 kB
          Andrew Wang
        7. HDFS-7960.003.patch
          46 kB
          Colin McCabe
        8. HDFS-7960.002.patch
          33 kB
          Colin McCabe

        Issue Links

          Activity

            People

              cmccabe Colin McCabe
              eddyxu Lei (Eddy) Xu
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: