[SAMZA-2405] Simplify Job Runner

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5
• Assignee: Ke Wu
• Reporter: Ke Wu
• Created: 27/Nov/19 18:35
• Updated: 26/Jul/21 22:43
• Resolved: 11/Jun/20 18:25

Description

Design: https://cwiki.apache.org/confluence/display/SAMZA/SEP-23%3A+Simplify+Job+Runner

Sub-Tasks

• Add ConfigLoader interface and default PropertiesConfigLoader implementation: Closed
• Add job.config.loader.* configs in JobConfig & corresponding env variables in ShellCommandConfig: Closed
• Update RemoteApplicationRunner to submit job only: Closed
• Update YarnJob to construct job submission env varible: Closed
• Update ClusterBasedJobCoordinator config retrieval logic to read config from loader: Closed
• Update CommandLine to use config loader for local config file: Closed
• Update LocalApplicationRunner to load full job config from loader: Closed
• Update ApplicationRunnerMain#ApplicationRunnerCommandLine not to load local file: Closed
• Update doc/tutorials from config factory to config loader: Closed
• Update ClusterBasedJobCoordinator to support Beam jobs: Resolved
• Update ProcessJobFactory and ThreadJobFactory to load full job config: Closed
• Simplify CommandLine: Closed
• Update ApplicationConfig to pass main class and arguments.: Resolved
• Add config file support in ApplicationRunnerMain to read submission config from a property file: Resolved
• Update doc/tutorials from config factory to config loader in samza-hello-samza example: Resolved
• Update JobCoordinatorLaunchUtil to fetch launch config from metadata store.: Resolved
• Fix JobCoordinatorLaunchUtil's execution sequence.: Closed
• Fix integration-tests: Closed