Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0-alpha
    • Component/s: tools
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      This is a new feature. It is documented in hdfs_user_guide.xml.

      Description

      When the NameNode metadata is corrupt for some reason, we want to be able to fix it. Obviously, we would prefer never to get in this case. In a perfect world, we never would. However, bad data on disk can happen from time to time, because of hardware errors or misconfigurations. In the past we have had to correct it manually, which is time-consuming and which can result in downtime.

      Recovery mode is initialized by the system administrator. When the NameNode starts up in Recovery Mode, it will try to load the FSImage file, apply all the edits from the edits log, and then write out a new image. Then it will shut down.

      Unlike in the normal startup process, the recovery mode startup process will be interactive. When the NameNode finds something that is inconsistent, it will prompt the operator as to what it should do. The operator can also choose to take the first option for all prompts by starting up with the '-f' flag, or typing 'a' at one of the prompts.

      I have reused as much code as possible from the NameNode in this tool. Hopefully, the effort that was spent developing this will also make the NameNode editLog and image processing even more robust than it already is.

        Attachments

        1. HDFS-3004.010.patch
          40 kB
          Colin P. McCabe
        2. HDFS-3004.011.patch
          56 kB
          Colin P. McCabe
        3. HDFS-3004.012.patch
          65 kB
          Colin P. McCabe
        4. HDFS-3004.013.patch
          64 kB
          Colin P. McCabe
        5. HDFS-3004.015.patch
          64 kB
          Colin P. McCabe
        6. HDFS-3004.016.patch
          65 kB
          Colin P. McCabe
        7. HDFS-3004.017.patch
          67 kB
          Colin P. McCabe
        8. HDFS-3004.018.patch
          89 kB
          Colin P. McCabe
        9. HDFS-3004.019.patch
          98 kB
          Colin P. McCabe
        10. HDFS-3004.020.patch
          101 kB
          Colin P. McCabe
        11. HDFS-3004.022.patch
          94 kB
          Colin P. McCabe
        12. HDFS-3004.023.patch
          95 kB
          Colin P. McCabe
        13. HDFS-3004.024.patch
          97 kB
          Colin P. McCabe
        14. HDFS-3004.026.patch
          100 kB
          Colin P. McCabe
        15. HDFS-3004.027.patch
          100 kB
          Colin P. McCabe
        16. HDFS-3004.029.patch
          100 kB
          Colin P. McCabe
        17. HDFS-3004.030.patch
          100 kB
          Colin P. McCabe
        18. HDFS-3004.031.patch
          100 kB
          Colin P. McCabe
        19. HDFS-3004.032.patch
          100 kB
          Colin P. McCabe
        20. HDFS-3004.033.patch
          100 kB
          Colin P. McCabe
        21. HDFS-3004.034.patch
          100 kB
          Colin P. McCabe
        22. HDFS-3004.035.patch
          100 kB
          Colin P. McCabe
        23. HDFS-3004.036.patch
          99 kB
          Colin P. McCabe
        24. HDFS-3004.037.patch
          100 kB
          Colin P. McCabe
        25. HDFS-3004.038.patch
          100 kB
          Colin P. McCabe
        26. HDFS-3004.039.patch
          101 kB
          Colin P. McCabe
        27. HDFS-3004.040.patch
          101 kB
          Colin P. McCabe
        28. HDFS-3004.041.patch
          101 kB
          Colin P. McCabe
        29. HDFS-3004__namenode_recovery_tool.txt
          3 kB
          Colin P. McCabe
        30. HDFS-3004.042.patch
          102 kB
          Colin P. McCabe
        31. HDFS-3004.042.patch
          102 kB
          Eli Collins
        32. HDFS-3004.042.patch
          102 kB
          Colin P. McCabe
        33. HDFS-3004.043.patch
          102 kB
          Colin P. McCabe
        34. recovery-mode.pdf
          99 kB
          Colin P. McCabe

          Issue Links

            Activity

              People

              • Assignee:
                cmccabe Colin P. McCabe
                Reporter:
                cmccabe Colin P. McCabe
              • Votes:
                0 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: