Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3002

HDFS should not remove blocks while in safemode.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.16.0
    • Fix Version/s: 0.17.2
    • Component/s: None
    • Labels:
      None

      Description

      I noticed that data-nodes are removing blocks during a rather prolonged distributed upgrade when the name-node is in safe mode.
      This happened on my experimental cluster with accelerated block report rate.
      By definition in safe mode the name-node should not

      • accept client requests to change the namespace state, and
      • schedule block replications and/or block removal for the data-nodes.

      We don't want any unnecessary replications until all blocks are reported during startup.
      We also don't want to remove blocks if safe mode is entered manually.
      In heartbeat processing we explicitly verify that the name-node is in safe-mode and do not return any block commands to the data-nodes.
      Block reports can also return block commands, which should be banned during safe mode.

        Attachments

        1. DelBlocksInSafeMode-017.patch
          7 kB
          Konstantin Shvachko
        2. DelBlocksInSafeMode-018.patch
          9 kB
          Konstantin Shvachko
        3. DelBlocksInSafeMode.patch
          8 kB
          Konstantin Shvachko
        4. DelBlocksInSafeMode-018.patch
          10 kB
          Konstantin Shvachko
        5. DelBlocksInSafeMode.patch
          11 kB
          Konstantin Shvachko

          Issue Links

            Activity

              People

              • Assignee:
                shv Konstantin Shvachko
                Reporter:
                shv Konstantin Shvachko
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: