Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-704

Create a tool to write coordinator stream

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: None
    • Labels:

      Description

      In order to be able to change configurations while a job is running, a tool for writing a message to the coordinator stream is needed. this task targets creating such a tool that can write messages to the coordinator stream after the bootstrap of the job.

      1. coordinator-stream-writer.patch
        27 kB
        Shadi A. Noghabi
      2. SAMZA-704-final.patch
        27 kB
        Navina Ramesh

        Activity

        Hide
        navina Navina Ramesh added a comment -

        Done!

        Show
        navina Navina Ramesh added a comment - Done!
        Hide
        closeuris Yan Fang added a comment -

        Sure. Just don't forget this.

        Show
        closeuris Yan Fang added a comment - Sure. Just don't forget this.
        Hide
        navina Navina Ramesh added a comment -

        I was thinking this can be part of SAMZA-614 - Document coordinator stream. ?? If not, I can create another JIRA for this.

        Show
        navina Navina Ramesh added a comment - I was thinking this can be part of SAMZA-614 - Document coordinator stream. ?? If not, I can create another JIRA for this.
        Hide
        closeuris Yan Fang added a comment -

        Can we have docs for the tool?

        Show
        closeuris Yan Fang added a comment - Can we have docs for the tool?
        Hide
        navina Navina Ramesh added a comment -

        Committed the patch.

        Shadi A. Noghabi : Good job! Apologize for forgetting to include you as author name in the commit.

        Show
        navina Navina Ramesh added a comment - Committed the patch. Shadi A. Noghabi : Good job! Apologize for forgetting to include you as author name in the commit.
        Hide
        navina Navina Ramesh added a comment -

        Attaching the final patch from the RB

        Show
        navina Navina Ramesh added a comment - Attaching the final patch from the RB
        Hide
        navina Navina Ramesh added a comment -

        Thanks for clarifying Shadi A. Noghabi ! Now it makes more sense

        Show
        navina Navina Ramesh added a comment - Thanks for clarifying Shadi A. Noghabi ! Now it makes more sense
        Hide
        closeuris Yan Fang added a comment -

        _ OK. If you are talking about this key, empty key makes sense.

        Show
        closeuris Yan Fang added a comment - _ OK. If you are talking about this key , empty key makes sense.
        Hide
        snoghabi Shadi A. Noghabi added a comment - - edited

        This key is not the key of the message it is the key in the key. That is ** key ** in key= [version, type, **key** ]

        Show
        snoghabi Shadi A. Noghabi added a comment - - edited This key is not the key of the message it is the key in the key. That is ** key ** in key= [version, type, **key** ]
        Hide
        snoghabi Shadi A. Noghabi added a comment - - edited

        If you look at design document at SAMZA-348, at the top of page 8, when you want to bounce a job, you send a message with type= restart-job, and empty key and value.

        Show
        snoghabi Shadi A. Noghabi added a comment - - edited If you look at design document at SAMZA-348 , at the top of page 8, when you want to bounce a job, you send a message with type= restart-job, and empty key and value.
        Hide
        closeuris Yan Fang added a comment -

        No, AFAIK. That is because the key always contains key =[version, type, key]. I do not think it has the meaning if the key is null.

        Show
        closeuris Yan Fang added a comment - No, AFAIK. That is because the key always contains key = [version, type, key] . I do not think it has the meaning if the key is null.
        Hide
        navina Navina Ramesh added a comment -

        Shadi A. Noghabi Left some comments in your RB.

        Looks like we are allowing null keys and null values in the coordinator stream messages. I can understand the null values. Do we have use-cases where the messages don't have keys, but only value? Yan Fang Yi Pan (Data Infrastructure) : Any thoughts?

        Show
        navina Navina Ramesh added a comment - Shadi A. Noghabi Left some comments in your RB. Looks like we are allowing null keys and null values in the coordinator stream messages. I can understand the null values. Do we have use-cases where the messages don't have keys, but only value? Yan Fang Yi Pan (Data Infrastructure) : Any thoughts?
        Hide
        gustavoanatoly Gustavo Anatoly added a comment -

        Hi, Shadi A. Noghabi

        I was taking a look in test case and I would like know if you tried use mockito instead using reflection java api.

        Cheers.

        Show
        gustavoanatoly Gustavo Anatoly added a comment - Hi, Shadi A. Noghabi I was taking a look in test case and I would like know if you tried use mockito instead using reflection java api. Cheers.
        Hide
        snoghabi Shadi A. Noghabi added a comment - - edited

        Patch for a tool to write to the coordinator stream added as attachment.

        Show
        snoghabi Shadi A. Noghabi added a comment - - edited Patch for a tool to write to the coordinator stream added as attachment.
        Hide
        snoghabi Shadi A. Noghabi added a comment -

        Thanks for adding the link of the RB. I will add the patch when ready.

        Show
        snoghabi Shadi A. Noghabi added a comment - Thanks for adding the link of the RB. I will add the patch when ready.
        Hide
        navina Navina Ramesh added a comment -

        For the benefit of others, this is the RB - https://reviews.apache.org/r/35241/

        Shadi A. Noghabi When you have addressed the comments in the RB, can you upload a patch here ?

        Show
        navina Navina Ramesh added a comment - For the benefit of others, this is the RB - https://reviews.apache.org/r/35241/ Shadi A. Noghabi When you have addressed the comments in the RB, can you upload a patch here ?

          People

          • Assignee:
            snoghabi Shadi A. Noghabi
            Reporter:
            snoghabi Shadi A. Noghabi
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development