Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1378

Edit log replay should track and report file offsets in case of errors

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.22.0
    • 0.23.0, 1.1.0
    • namenode
    • None
    • Reviewed

    Description

      Occasionally there are bugs or operational mistakes that result in corrupt edit logs which I end up having to repair by hand. In these cases it would be very handy to have the error message also print out the file offsets of the last several edit log opcodes so it's easier to find the right place to edit in the OP_INVALID marker. We could also use this facility to provide a rough estimate of how far along edit log replay the NN is during startup (handy when a 2NN has died and replay takes a while)

      Attachments

        1. hdfs-1378.0.patch
          36 kB
          Aaron Myers
        2. hdfs-1378.1.patch
          37 kB
          Aaron Myers
        3. hdfs-1378.2.txt
          38 kB
          Todd Lipcon
        4. HDFS-1378-b1.002.patch
          4 kB
          Colin McCabe
        5. HDFS-1378-b1.003.patch
          5 kB
          Colin McCabe
        6. HDFS-1378-b1.004.patch
          9 kB
          Colin McCabe
        7. hdfs-1378-branch20.txt
          4 kB
          Todd Lipcon

        Issue Links

          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. HDFS-3055 Implement recovery mode for branch-1

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              cmccabe Colin McCabe
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: