Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5428

under construction files deletion after snapshot+checkpoint+nn restart leads nn safemode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.3.0
    • snapshots
    • None
    • Reviewed

    Description

      1. allow snapshots under dir /foo
      2. create a file /foo/test/bar and start writing to it
      3. create a snapshot s1 under /foo after block is allocated and some data has been written to it
      4. Delete the directory /foo/test
      5. wait till checkpoint or do saveNameSpace
      6. restart NN.

      NN enters to safemode.

      Analysis:
      Snapshot nodes loaded from fsimage are always complete and all blocks will be in COMPLETE state.
      So when the Datanode reports RBW blocks those will not be updated in blocksmap.
      Some of the FINALIZED blocks will be marked as corrupt due to length mismatch.

      Attachments

        1. HDFS-5428.000.patch
          19 kB
          Jing Zhao
        2. HDFS-5428.001.patch
          21 kB
          Jing Zhao
        3. HDFS-5428.002.patch
          24 kB
          Jing Zhao
        4. HDFS-5428.003.patch
          22 kB
          Jing Zhao
        5. HDFS-5428.004.patch
          21 kB
          Jing Zhao
        6. HDFS-5428.patch
          10 kB
          Vinayakumar B
        7. HDFS-5428-v2.patch
          84 kB
          Vinayakumar B

        Issue Links

          Activity

            People

              jingzhao Jing Zhao
              vinayakumarb Vinayakumar B
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: