Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10641

Introduce Coordination Engine interface

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.0.0-alpha1
    • None
    • None
    • None

    Description

      Coordination Engine (CE) is a system, which allows to agree on a sequence of events in a distributed system. In order to be reliable CE should be distributed by itself.
      Coordination Engine can be based on different algorithms (paxos, raft, 2PC, zab) and have different implementations, depending on use cases, reliability, availability, and performance requirements.
      CE should have a common API, so that it could serve as a pluggable component in different projects. The immediate beneficiaries are HDFS (HDFS-6469) and HBase (HBASE-10909).
      First implementation is proposed to be based on ZooKeeper.

      Attachments

        1. zkCEBenchmark.pdf
          46 kB
          Andrey Stepachev
        2. zkbench.pdf
          49 kB
          Henry Wang
        3. NNThroughputBenchmark Results.pdf
          61 kB
          Henry Wang
        4. hadoop-coordination.patch
          63 kB
          Mikhail Antonov
        5. HADOOP-10641.patch
          60 kB
          Konstantin I Boudnik
        6. HADOOP-10641.patch
          75 kB
          Plamen Jeliazkov
        7. HADOOP-10641.patch
          60 kB
          Plamen Jeliazkov
        8. HADOOP-10641.patch
          72 kB
          Plamen Jeliazkov
        9. HADOOP-10641.patch
          65 kB
          Plamen Jeliazkov
        10. ce-tla.zip
          38 kB
          Michael Parkin

        Issue Links

          blocks

          Umbrella - An overarching type made of sub-tasks HBASE-11241 Support for multiple active HMaster replicas

          • Major - Major loss of function.
          • Closed
          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. HDFS-6469 Coordinated replication of the namespace using ConsensusNode

          • Major - Major loss of function.
          • Open
          links to

          Web Link CNode Deep Dive meetup slides

          Web Link Coordination Engine described in ConsensusNode design document

          Activity

            People

              zero45 Plamen Jeliazkov
              shv Konstantin Shvachko
              Votes:
              2 Vote for this issue
              Watchers:
              48 Start watching this issue

              Dates

                Created:
                Updated: