Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-2405

Simplify Job Runner

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5
    • Component/s: None
    • Labels:
      None

      Attachments

      1.
      Add ConfigLoader interface and default PropertiesConfigLoader implementation Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 10h 10m
      Actions
      2.
      Add job.config.loader.* configs in JobConfig & corresponding env variables in ShellCommandConfig Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 3h 40m
      Actions
      3.
      Update RemoteApplicationRunner to submit job only Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 2h 50m
      Actions
      4.
      Update YarnJob to construct job submission env varible Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 50m
      Actions
      5.
      Update ClusterBasedJobCoordinator config retrieval logic to read config from loader Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 5.5h
      Actions
      6.
      Update CommandLine to use config loader for local config file Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 3h 50m
      Actions
      7.
      Update LocalApplicationRunner to load full job config from loader Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 3h
      Actions
      8.
      Update ApplicationRunnerMain#ApplicationRunnerCommandLine not to load local file Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 1h 10m
      Actions
      9.
      Update doc/tutorials from config factory to config loader Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 1h 20m
      Actions
      10.
      Update ClusterBasedJobCoordinator to support Beam jobs Sub-task Resolved Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 1h
      Actions
      11.
      Update ProcessJobFactory and ThreadJobFactory to load full job config Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 2h
      Actions
      12.
      Simplify CommandLine Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 20m
      Actions
      13.
      Update ApplicationConfig to pass main class and arguments. Sub-task Resolved Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 20m
      Actions
      14.
      Add config file support in ApplicationRunnerMain to read submission config from a property file Sub-task Resolved Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 20m
      Actions
      15.
      Update doc/tutorials from config factory to config loader in samza-hello-samza example Sub-task Resolved Unassigned   Actions
      16.
      Update JobCoordinatorLaunchUtil to fetch launch config from metadata store. Sub-task Resolved Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 1h 50m
      Actions
      17.
      Fix JobCoordinatorLaunchUtil's execution sequence. Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 1h 50m
      Actions
      18.
      Fix integration-tests Sub-task Closed Unassigned

      100%

      Original Estimate - Not Specified Original Estimate - Not Specified
      Time Spent - 20m
      Actions

        Activity

          People

          • Assignee:
            kw2542 Ke Wu
            Reporter:
            kw2542 Ke Wu

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 40h 20m
              40h 20m

                Issue deployment