Description
This is an exploratory ticket to figure out if we want to support topologies in Samza.
Potential solutions include:
1. Don't support them as part of the framework (a la Hadoop).
2. Use Tez as topology runner.
3. Use config inheritence (.properties file merging). This already exists.
4. Write some kind of JobGroupRunner that executes multiple jobs at one time.
5. ?
There are probably a lot of solutions. Some goals of this ticket are:
a. Config sharing between job with a lot of the same config.
b. Whether we want to run multiple jobs with a single AM (in YARN).
c. How to start/stop only a subset of job when running in a topology.
Attachments
Issue Links
- relates to
-
SAMZA-300 Track producers and consumers of streams
- Open