Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3077

Quorum-based protocol for reading and writing edit logs

    Details

      Description

      Currently, one of the weak points of the HA design is that it relies on shared storage such as an NFS filer for the shared edit log. One alternative that has been proposed is to depend on BookKeeper, a ZooKeeper subproject which provides a highly available replicated edit log on commodity hardware. This JIRA is to implement another alternative, based on a quorum commit protocol, integrated more tightly in HDFS and with the requirements driven only by HDFS's needs rather than more generic use cases. More details to follow.

      1. hdfs-3077-partial.txt
        110 kB
        Todd Lipcon
      2. qjournal-design.pdf
        229 kB
        Todd Lipcon
      3. qjournal-design.pdf
        251 kB
        Todd Lipcon
      4. hdfs-3077.txt
        209 kB
        Todd Lipcon
      5. hdfs-3077.txt
        214 kB
        Todd Lipcon
      6. hdfs-3077.txt
        231 kB
        Todd Lipcon
      7. hdfs-3077.txt
        239 kB
        Todd Lipcon
      8. hdfs-3077.txt
        239 kB
        Todd Lipcon
      9. hdfs-3077.txt
        239 kB
        Todd Lipcon
      10. hdfs-3077.txt
        239 kB
        Todd Lipcon
      11. qjournal-design.pdf
        275 kB
        Todd Lipcon
      12. qjournal-design.tex
        43 kB
        Todd Lipcon
      13. hdfs-3077-test-merge.txt
        525 kB
        Todd Lipcon
      14. qjournal-design.tex
        48 kB
        Todd Lipcon
      15. qjournal-design.pdf
        285 kB
        Todd Lipcon
      16. qjournal-design.pdf
        287 kB
        Todd Lipcon
      17. qjournal-design.pdf
        293 kB
        Todd Lipcon
      18. hdfs-3077-branch-2.txt
        527 kB
        Todd Lipcon

        Issue Links

          Activity

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              83 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development