Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-955

FSImage.saveFSImage can lose edits

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 0.20.1, 0.21.0, 0.22.0
    • 0.20.3
    • namenode
    • None
    • Reviewed

    Description

      This is a continuation of a discussion from HDFS-909. The FSImage.saveFSImage function (implementing dfsadmin -saveNamespace) can corrupt the NN storage such that all current edits are lost.

      Attachments

        1. saveNamespace-0.20.patch
          37 kB
          Konstantin Shvachko
        2. saveNamespace-0.21.patch
          41 kB
          Konstantin Shvachko
        3. saveNamespace.patch
          41 kB
          Konstantin Shvachko
        4. saveNamespace.patch
          41 kB
          Konstantin Shvachko
        5. saveNamespace.patch
          38 kB
          Konstantin Shvachko
        6. saveNamespace-0.20.patch
          30 kB
          Konstantin Shvachko
        7. saveNamespace-0.21.patch
          34 kB
          Konstantin Shvachko
        8. saveNamespace.patch
          34 kB
          Konstantin Shvachko
        9. saveNamespace.txt
          33 kB
          Konstantin Shvachko
        10. FSStateTransition7.htm
          119 kB
          Konstantin Shvachko
        11. hdfs-955-moretests.txt
          16 kB
          Todd Lipcon
        12. PurgeEditsBeforeImageSave.patch
          3 kB
          Konstantin Shvachko
        13. hdfs-955-unittest.txt
          5 kB
          Todd Lipcon

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-909 Race condition between rollEditLog or rollFSImage ant FSEditsLog.write operations corrupts edits log

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-87 NameNode startup fails if edit log terminates prematurely

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-988 saveNamespace race can corrupt the edits log

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-970 FSImage writing should always fsync before close

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-957 FSImage layout version should be only once file is complete

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-1073 Simpler model for Namenode's fs Image and edit Logs

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              shv Konstantin Shvachko
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: