Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-73

Add topologies to Samza

    XMLWordPrintableJSON

    Details

    • Type: Wish
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.6.0
    • Fix Version/s: None
    • Component/s: build, container, yarn
    • Labels:

      Description

      This is an exploratory ticket to figure out if we want to support topologies in Samza.

      Potential solutions include:

      1. Don't support them as part of the framework (a la Hadoop).
      2. Use Tez as topology runner.
      3. Use config inheritence (.properties file merging). This already exists.
      4. Write some kind of JobGroupRunner that executes multiple jobs at one time.
      5. ?

      There are probably a lot of solutions. Some goals of this ticket are:

      a. Config sharing between job with a lot of the same config.
      b. Whether we want to run multiple jobs with a single AM (in YARN).
      c. How to start/stop only a subset of job when running in a topology.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                criccomini Chris Riccomini
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: