Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-516

Support standalone Samza jobs



    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0
    • Fix Version/s: None
    • Component/s: container
    • Labels:


      Samza currently supports two modes of operation out of the box: local and YARN. With local mode, a single Java process starts the JobCoordinator, creates a single container, and executes it locally. All partitions are procesed within this container. With YARN, a YARN grid is required to execute the Samza job. In addition, SAMZA-375 introduces a patch to run Samza in Mesos.

      There have been several requests lately to be able to run Samza jobs without any resource manager (YARN, Mesos, etc), but still run it in a distributed fashion.

      The goal of this ticket is to design and implement a samza-standalone module, which will:

      1. Support executing a single Samza job in one or more containers.
      2. Support failover, in cases where a machine is lost.


        1. DESIGN-SAMZA-516-2.pdf
          293 kB
          Chris Riccomini
        2. DESIGN-SAMZA-516-2.md
          28 kB
          Chris Riccomini
        3. DESIGN-SAMZA-516-1.pdf
          177 kB
          Chris Riccomini
        4. DESIGN-SAMZA-516-1.md
          15 kB
          Chris Riccomini
        5. DESIGN-SAMZA-516-0.pdf
          139 kB
          Chris Riccomini
        6. DESIGN-SAMZA-516-0.md
          10 kB
          Chris Riccomini

          Issue Links



              • Assignee:
                criccomini Chris Riccomini
                criccomini Chris Riccomini
              • Votes:
                3 Vote for this issue
                18 Start watching this issue


                • Created: