Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0-alpha
    • Component/s: tools
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      This is a new feature. It is documented in hdfs_user_guide.xml.

      Description

      When the NameNode metadata is corrupt for some reason, we want to be able to fix it. Obviously, we would prefer never to get in this case. In a perfect world, we never would. However, bad data on disk can happen from time to time, because of hardware errors or misconfigurations. In the past we have had to correct it manually, which is time-consuming and which can result in downtime.

      Recovery mode is initialized by the system administrator. When the NameNode starts up in Recovery Mode, it will try to load the FSImage file, apply all the edits from the edits log, and then write out a new image. Then it will shut down.

      Unlike in the normal startup process, the recovery mode startup process will be interactive. When the NameNode finds something that is inconsistent, it will prompt the operator as to what it should do. The operator can also choose to take the first option for all prompts by starting up with the '-f' flag, or typing 'a' at one of the prompts.

      I have reused as much code as possible from the NameNode in this tool. Hopefully, the effort that was spent developing this will also make the NameNode editLog and image processing even more robust than it already is.

      1. HDFS-3004.010.patch
        40 kB
        Colin Patrick McCabe
      2. HDFS-3004.011.patch
        56 kB
        Colin Patrick McCabe
      3. HDFS-3004.012.patch
        65 kB
        Colin Patrick McCabe
      4. HDFS-3004.013.patch
        64 kB
        Colin Patrick McCabe
      5. HDFS-3004.015.patch
        64 kB
        Colin Patrick McCabe
      6. HDFS-3004.016.patch
        65 kB
        Colin Patrick McCabe
      7. HDFS-3004.017.patch
        67 kB
        Colin Patrick McCabe
      8. HDFS-3004.018.patch
        89 kB
        Colin Patrick McCabe
      9. HDFS-3004.019.patch
        98 kB
        Colin Patrick McCabe
      10. HDFS-3004.020.patch
        101 kB
        Colin Patrick McCabe
      11. HDFS-3004.022.patch
        94 kB
        Colin Patrick McCabe
      12. HDFS-3004.023.patch
        95 kB
        Colin Patrick McCabe
      13. HDFS-3004.024.patch
        97 kB
        Colin Patrick McCabe
      14. HDFS-3004.026.patch
        100 kB
        Colin Patrick McCabe
      15. HDFS-3004.027.patch
        100 kB
        Colin Patrick McCabe
      16. HDFS-3004.029.patch
        100 kB
        Colin Patrick McCabe
      17. HDFS-3004.030.patch
        100 kB
        Colin Patrick McCabe
      18. HDFS-3004.031.patch
        100 kB
        Colin Patrick McCabe
      19. HDFS-3004.032.patch
        100 kB
        Colin Patrick McCabe
      20. HDFS-3004.033.patch
        100 kB
        Colin Patrick McCabe
      21. HDFS-3004.034.patch
        100 kB
        Colin Patrick McCabe
      22. HDFS-3004.035.patch
        100 kB
        Colin Patrick McCabe
      23. HDFS-3004.036.patch
        99 kB
        Colin Patrick McCabe
      24. HDFS-3004.037.patch
        100 kB
        Colin Patrick McCabe
      25. HDFS-3004.038.patch
        100 kB
        Colin Patrick McCabe
      26. HDFS-3004.039.patch
        101 kB
        Colin Patrick McCabe
      27. HDFS-3004.040.patch
        101 kB
        Colin Patrick McCabe
      28. HDFS-3004.041.patch
        101 kB
        Colin Patrick McCabe
      29. HDFS-3004__namenode_recovery_tool.txt
        3 kB
        Colin Patrick McCabe
      30. HDFS-3004.042.patch
        102 kB
        Colin Patrick McCabe
      31. HDFS-3004.042.patch
        102 kB
        Eli Collins
      32. HDFS-3004.042.patch
        102 kB
        Colin Patrick McCabe
      33. HDFS-3004.043.patch
        102 kB
        Colin Patrick McCabe
      34. recovery-mode.pdf
        99 kB
        Colin Patrick McCabe

        Issue Links

          Activity

            People

            • Assignee:
              Colin Patrick McCabe
              Reporter:
              Colin Patrick McCabe
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development