Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-5

AdaptorId should be persistent even after an agent restart

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.2.0
    • Component/s: Data Collection
    • Labels:
      None

      Description

      In case of agent restart, an adaptor may not get the same adaptorId.

      1)The adaptorId is sent back to the remote application after the Add command.
      Now, let suppose the agent dies and restarts.
      Each new adaptor created from the checkpoint will now have another adaptorId
      .
      If the remote application tries to stop/shutdown only using the adaptorId received in step (1), the command may be sent to the wrong adaptor.

      if the adaptorId in for example an MD5 of the command line we don't need to save any additional information inside the checkpoint.
      This will change the current API from int to String.

      1. CHUKWA-5.patch
        58 kB
        Ari Rabkin

        Issue Links

          Activity

          Hide
          asrabkin Ari Rabkin added a comment -

          An alternative to using MD5 would be to store the adaptor ID in the checkpoint file.
          This would probably require modifying checkpointing to use some new command, rather than "add" that explicitly sets the adaptor ID. This is probably not to hard.

          The advantage is that it's much easier for a human to reason about small integer IDs than biggish MD5 hashes.

          Show
          asrabkin Ari Rabkin added a comment - An alternative to using MD5 would be to store the adaptor ID in the checkpoint file. This would probably require modifying checkpointing to use some new command, rather than "add" that explicitly sets the adaptor ID. This is probably not to hard. The advantage is that it's much easier for a human to reason about small integer IDs than biggish MD5 hashes.
          Hide
          asrabkin Ari Rabkin added a comment -

          I've been thinking about this.
          I think the real fix is actually to give adaptors names, not numbers at all. And then insist that names be unique.

          Show
          asrabkin Ari Rabkin added a comment - I've been thinking about this. I think the real fix is actually to give adaptors names, not numbers at all. And then insist that names be unique.
          Hide
          jboulon Jerome Boulon added a comment -

          I come back with my MD5, ..., if we're doing an MD5 of the command it should be unique and that could be the name from our log4j Appender (automatically done for you) and from the command line user are still free to put a more descriptive name.

          Show
          jboulon Jerome Boulon added a comment - I come back with my MD5, ..., if we're doing an MD5 of the command it should be unique and that could be the name from our log4j Appender (automatically done for you) and from the command line user are still free to put a more descriptive name.
          Hide
          asrabkin Ari Rabkin added a comment -

          Is this on the roadmap for 0.2? It's going to be a backwards-incompatible change to the agent wire protocol, checkpoints, and initial_adaptors, I'm afraid, maybe more. And so I'd like to bite the bullet and do it sooner, rather than later.

          Show
          asrabkin Ari Rabkin added a comment - Is this on the roadmap for 0.2? It's going to be a backwards-incompatible change to the agent wire protocol, checkpoints, and initial_adaptors, I'm afraid, maybe more. And so I'd like to bite the bullet and do it sooner, rather than later.
          Hide
          asrabkin Ari Rabkin added a comment -

          Just talked to Mac and Jerome. Conclusions:

          • Agent datastructures will use strings to identify adaptors. Client will use MD5.
          • Gonna push this into 0.2, but patch will likely happen post-branch.
          Show
          asrabkin Ari Rabkin added a comment - Just talked to Mac and Jerome. Conclusions: Agent datastructures will use strings to identify adaptors. Client will use MD5. Gonna push this into 0.2, but patch will likely happen post-branch.
          Hide
          asrabkin Ari Rabkin added a comment -

          Proposed new Agent-side add command syntax:
          "Add name=...."
          Where name is a string which doesn't begin or end with whitespace, and doesn't contain an =.

          Alternatively, the traditional syntax will continue to work, in which case the name will be autogenerated, via MD5 of the parameters.

          Show
          asrabkin Ari Rabkin added a comment - Proposed new Agent-side add command syntax: "Add name=...." Where name is a string which doesn't begin or end with whitespace, and doesn't contain an =. Alternatively, the traditional syntax will continue to work, in which case the name will be autogenerated, via MD5 of the parameters.
          Hide
          asrabkin Ari Rabkin added a comment -

          Change adaptor IDs to strings. Default to MD5 of arguments.
          Propose to commit this to both trunk and 0.2

          Note. This is an INCOMPATIBLE change in many ways.

          Show
          asrabkin Ari Rabkin added a comment - Change adaptor IDs to strings. Default to MD5 of arguments. Propose to commit this to both trunk and 0.2 Note. This is an INCOMPATIBLE change in many ways.
          Hide
          macyang Mac Yang added a comment -

          +1 on getting this into 0.2

          Show
          macyang Mac Yang added a comment - +1 on getting this into 0.2
          Hide
          eyang Eric Yang added a comment -

          +1 looks good.

          Show
          eyang Eric Yang added a comment - +1 looks good.
          Hide
          asrabkin Ari Rabkin added a comment -

          Stuff will probably break, but here goes!

          Show
          asrabkin Ari Rabkin added a comment - Stuff will probably break, but here goes!
          Hide
          hudson Hudson added a comment -
          Show
          hudson Hudson added a comment - Integrated in Chukwa-trunk #53 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/53/ )

            People

            • Assignee:
              asrabkin Ari Rabkin
              Reporter:
              jboulon Jerome Boulon
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development