Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2003

Separate FSEditLog reading logic from editLog memory state building logic

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • Edit log branch (HDFS-1073), 0.23.0
    • Edit log branch (HDFS-1073), 0.23.0
    • None
    • None
    • Reviewed

    Description

      Currently FSEditLogLoader has code for reading from an InputStream interleaved with code which updates the FSNameSystem and FSDirectory. This makes it difficult to read an edit log without having a whole load of other object initialised, which is problematic if you want to do things like count how many transactions are in a file etc.

      This patch separates the reading of the stream and the building of the memory state.

      Attachments

        1. HDFS-2003-replicationfix-delta.diff
          3 kB
          Ivan Kelly
        2. hdfs-2003.txt
          50 kB
          Todd Lipcon
        3. hdfs-2003.txt
          51 kB
          Todd Lipcon
        4. hdfs-2003.txt
          54 kB
          Todd Lipcon
        5. HDFS-2003.diff
          76 kB
          Ivan Kelly
        6. HDFS-2003.diff
          78 kB
          Ivan Kelly
        7. HDFS-2003.diff
          79 kB
          Ivan Kelly
        8. HDFS-2003.diff
          54 kB
          Ivan Kelly
        9. HDFS-2003.diff
          51 kB
          Ivan Kelly
        10. HDFS-2003.diff
          54 kB
          Ivan Kelly
        11. HDFS-2003.diff
          54 kB
          Ivan Kelly
        12. HDFS-2003.diff
          54 kB
          Ivan Kelly
        13. 2003-delta.txt
          6 kB
          Todd Lipcon

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-1580 Add interface for generic Write Ahead Logging mechanisms

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-2041 Some mtimes and atimes are lost when edit logs are replayed

          • Major - Major loss of function.
          • Closed

          Activity

            People

              ikelly Ivan Kelly
              ikelly Ivan Kelly
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: