Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5428

under construction files deletion after snapshot+checkpoint+nn restart leads nn safemode

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: snapshots
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      1. allow snapshots under dir /foo
      2. create a file /foo/test/bar and start writing to it
      3. create a snapshot s1 under /foo after block is allocated and some data has been written to it
      4. Delete the directory /foo/test
      5. wait till checkpoint or do saveNameSpace
      6. restart NN.

      NN enters to safemode.

      Analysis:
      Snapshot nodes loaded from fsimage are always complete and all blocks will be in COMPLETE state.
      So when the Datanode reports RBW blocks those will not be updated in blocksmap.
      Some of the FINALIZED blocks will be marked as corrupt due to length mismatch.

        Attachments

        1. HDFS-5428-v2.patch
          84 kB
          Vinayakumar B
        2. HDFS-5428.patch
          10 kB
          Vinayakumar B
        3. HDFS-5428.004.patch
          21 kB
          Jing Zhao
        4. HDFS-5428.003.patch
          22 kB
          Jing Zhao
        5. HDFS-5428.002.patch
          24 kB
          Jing Zhao
        6. HDFS-5428.001.patch
          21 kB
          Jing Zhao
        7. HDFS-5428.000.patch
          19 kB
          Jing Zhao

          Issue Links

            Activity

              People

              • Assignee:
                jingzhao Jing Zhao
                Reporter:
                vinayakumarb Vinayakumar B
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: