Hadoop Common
  1. Hadoop Common
  2. HADOOP-4935

Manual leaving of safe mode may lead to data lost

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.18.3
    • Fix Version/s: 0.18.3
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Due to HADOOP-4610, NameNode calculates mis-replicated blocks when leaving safe mode manually, where it clears the pending deletion queue before it does the calculation. This works fine when NameNode just starts but introduced a bug when NameNode is running for a while. Clearing the pending deletion queue makes NameNode not able to distinguish valid replicas from invalid ones, ie, the ones that have scheduled or dispatched for deletion. Therefore, NameNode may mistakenly decide the block is over-replicated and choose all valid ones to delete.

      1. misReplBlocks-0-18.patch
        0.6 kB
        Konstantin Shvachko
      2. misReplBlocks.patch
        0.7 kB
        Konstantin Shvachko

        Activity

        Hairong Kuang created issue -
        Konstantin Shvachko made changes -
        Field Original Value New Value
        Attachment misReplBlocks.patch [ 12396705 ]
        Konstantin Shvachko made changes -
        Attachment misReplBlocks-0-18.patch [ 12396706 ]
        Konstantin Shvachko made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Konstantin Shvachko made changes -
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Nigel Daley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Owen O'Malley made changes -
        Component/s dfs [ 12310710 ]

          People

          • Assignee:
            Konstantin Shvachko
            Reporter:
            Hairong Kuang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development