Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3335

check for edit log corruption at the end of the log

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Even after encountering an OP_INVALID, we should check the end of the edit log to make sure that it contains no more edits.

      This will catch things like rare race conditions or log corruptions that would otherwise remain undetected. They will got from being silent data loss scenarios to being cases that we can detect and fix.

      Using recovery mode, we can choose to ignore the end of the log if necessary.

        Attachments

        1. HDFS-3335.011.patch
          25 kB
          Colin P. McCabe
        2. HDFS-3335.010.patch
          24 kB
          Colin P. McCabe
        3. HDFS-3335.009.patch
          24 kB
          Colin P. McCabe
        4. HDFS-3335.008.patch
          23 kB
          Colin P. McCabe
        5. HDFS-3335-b1.004.patch
          11 kB
          Colin P. McCabe
        6. HDFS-3335.007.patch
          25 kB
          Colin P. McCabe
        7. HDFS-3335.006.patch
          25 kB
          Colin P. McCabe
        8. HDFS-3335.005.patch
          24 kB
          Colin P. McCabe
        9. HDFS-3335.004.patch
          24 kB
          Colin P. McCabe
        10. HDFS-3335.003.patch
          23 kB
          Colin P. McCabe
        11. HDFS-3335.002.patch
          23 kB
          Colin P. McCabe
        12. HDFS-3335.001.patch
          9 kB
          Colin P. McCabe
        13. HDFS-3335-b1.003.patch
          10 kB
          Colin P. McCabe
        14. HDFS-3335-b1.002.patch
          10 kB
          Colin P. McCabe
        15. HDFS-3335-b1.001.patch
          10 kB
          Colin P. McCabe

          Issue Links

            Activity

              People

              • Assignee:
                cmccabe Colin P. McCabe
                Reporter:
                cmccabe Colin P. McCabe
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: