Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-955

FSImage.saveFSImage can lose edits

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 0.20.1, 0.21.0, 0.22.0
    • 0.20.3
    • namenode
    • None
    • Reviewed

    Description

      This is a continuation of a discussion from HDFS-909. The FSImage.saveFSImage function (implementing dfsadmin -saveNamespace) can corrupt the NN storage such that all current edits are lost.

      Attachments

        1. FSStateTransition7.htm
          119 kB
          Konstantin Shvachko
        2. hdfs-955-moretests.txt
          16 kB
          Todd Lipcon
        3. hdfs-955-unittest.txt
          5 kB
          Todd Lipcon
        4. PurgeEditsBeforeImageSave.patch
          3 kB
          Konstantin Shvachko
        5. saveNamespace.patch
          41 kB
          Konstantin Shvachko
        6. saveNamespace.patch
          41 kB
          Konstantin Shvachko
        7. saveNamespace.patch
          38 kB
          Konstantin Shvachko
        8. saveNamespace.patch
          34 kB
          Konstantin Shvachko
        9. saveNamespace.txt
          33 kB
          Konstantin Shvachko
        10. saveNamespace-0.20.patch
          37 kB
          Konstantin Shvachko
        11. saveNamespace-0.20.patch
          30 kB
          Konstantin Shvachko
        12. saveNamespace-0.21.patch
          41 kB
          Konstantin Shvachko
        13. saveNamespace-0.21.patch
          34 kB
          Konstantin Shvachko

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-909 Race condition between rollEditLog or rollFSImage ant FSEditsLog.write operations corrupts edits log

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-87 NameNode startup fails if edit log terminates prematurely

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-988 saveNamespace race can corrupt the edits log

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-970 FSImage writing should always fsync before close

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-957 FSImage layout version should be only once file is complete

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-1073 Simpler model for Namenode's fs Image and edit Logs

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              shv Konstantin Shvachko
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: