Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-849

Split-deployment of samza framework

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:

      Description

      To allow separate deployment of Samza framework from user job's code.
      See design doc for details.

      1. split_deploy_os.patch
        19 kB
        Boris Shkolnik
      2. split_deploy_os2.patch
        20 kB
        Boris Shkolnik
      3. SplitDeploymentDesign.md
        8 kB
        Boris Shkolnik
      4. SplitDeploymentDesign.md
        6 kB
        Boris Shkolnik
      5. SplitDeploymentDesign.pdf
        66 kB
        Boris Shkolnik
      6. SplitDeploymentDesignProposal1.pdf
        68 kB
        Boris Shkolnik

        Activity

        Hide
        boryas Boris Shkolnik added a comment -

        Attaching design docs.

        Show
        boryas Boris Shkolnik added a comment - Attaching design docs.
        Hide
        xinyu Xinyu Liu added a comment -

        The design doc looks clear to me. The samza lib and user jars are compiled and deployed separately, and combined together during runtime. The framework path setup looks reasonable to me. It allows canary and rollback of different Samza framework versions during the upgrade. One comment: could you please comment on developing locally on split deployment, say, using ProcessJobFactory or ThreadJobFactory? It's not mentioned in the design doc. Thanks.

        Show
        xinyu Xinyu Liu added a comment - The design doc looks clear to me. The samza lib and user jars are compiled and deployed separately, and combined together during runtime. The framework path setup looks reasonable to me. It allows canary and rollback of different Samza framework versions during the upgrade. One comment: could you please comment on developing locally on split deployment, say, using ProcessJobFactory or ThreadJobFactory? It's not mentioned in the design doc. Thanks.
        Hide
        nickpan47 Yi Pan (Data Infrastructure) added a comment -

        +1 on the overall design. It would good to relate this w/ SAMZA-697 as well.

        Adding to Xinyu Liu's comment, it would be good to describe the usage/configuration for all deployment scenarios as well.

        Show
        nickpan47 Yi Pan (Data Infrastructure) added a comment - +1 on the overall design. It would good to relate this w/ SAMZA-697 as well. Adding to Xinyu Liu 's comment, it would be good to describe the usage/configuration for all deployment scenarios as well.
        Hide
        boryas Boris Shkolnik added a comment - - edited

        Addressed comments from Yi Pan (Data Infrastructure) and xiliu.
        Added suggestion for local deployment and info about per job ClassLoader.

        Show
        boryas Boris Shkolnik added a comment - - edited Addressed comments from Yi Pan (Data Infrastructure) and xiliu . Added suggestion for local deployment and info about per job ClassLoader.
        Show
        boryas Boris Shkolnik added a comment - https://reviews.apache.org/r/44604/
        Hide
        boryas Boris Shkolnik added a comment -

        uploaded the patch.

        Show
        boryas Boris Shkolnik added a comment - uploaded the patch.
        Hide
        boryas Boris Shkolnik added a comment -

        merged with the master.

        Show
        boryas Boris Shkolnik added a comment - merged with the master.
        Hide
        navina Navina Ramesh added a comment -

        Thanks for the updated patch, Boris Shkolnik ! I have committed it. We can resolve this feature JIRA once the documentation is complete. Thanks!

        Show
        navina Navina Ramesh added a comment - Thanks for the updated patch, Boris Shkolnik ! I have committed it. We can resolve this feature JIRA once the documentation is complete. Thanks!

          People

          • Assignee:
            boryas Boris Shkolnik
            Reporter:
            boryas Boris Shkolnik
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development