Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-516 Support standalone Samza jobs
  3. SAMZA-555

Write a ZK-based samza-standalone JobRunner

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0
    • Fix Version/s: None
    • Component/s: container
    • Labels:
      None

      Description

      SAMZA-516 documents the design of a standalone Samza job, which uses ZK for liveness monitoring and partition assignment rather than YARN. The first step to implementing this design is to add a samza-standalone submodule in Samza, which implements a JobRunner that uses ZK to elect a leader for the job (JobCoordinator), and then start a container. When failures occur, the leader should shift partitions within ZK to move partitions off of the dead container. Likewise, when a new container is added, the leader should shift partitions to the new container. The SamzaContainer should remain immutable, and simply be bounced when partitions are added or removed from it.

        Attachments

        1. SAMZA-555-1.patch
          56 kB
          Chris Riccomini
        2. SAMZA-555-0.patch
          53 kB
          Chris Riccomini

          Issue Links

            Activity

              People

              • Assignee:
                criccomini Chris Riccomini
                Reporter:
                criccomini Chris Riccomini
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: