Chukwa
  1. Chukwa
  2. CHUKWA-284

division of responsibility between agent and adaptor

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.1.2, 0.2.0
    • Fix Version/s: 0.3.0
    • Component/s: Data Collection
    • Labels:
      None

      Description

      The division of responsibility between adaptor and agent for managing offsets, datatypes, and such has gotten quite squirrely. And the comments don't match the code.

      For instance, the adaptors track last offset. This means that checkpoint files include the last byte enqueued to be SENT, rather than the last byte ACKNOWLEDGED. This violates both the spec and the comments.

      I propose that we make the agent responsible for recording offsets and possibly also datatypes. Are there cases where an Adaptor decides on its own what the Datatype is for the chunks it outputs? Or outputs chunks of different Types?

        Activity

        Hide
        Ari Rabkin added a comment -

        This isn't going to be fixed in 0.2

        Show
        Ari Rabkin added a comment - This isn't going to be fixed in 0.2
        Hide
        Ari Rabkin added a comment -

        Should only checkpoint adaptor status once we got an ack back from collector. This patch does that.

        Show
        Ari Rabkin added a comment - Should only checkpoint adaptor status once we got an ack back from collector. This patch does that.
        Hide
        Ari Rabkin added a comment -

        Change to chukwa-config.sh does not belong in this patch. However, barring objection, I'm going to commit the rest of it.

        Show
        Ari Rabkin added a comment - Change to chukwa-config.sh does not belong in this patch. However, barring objection, I'm going to commit the rest of it.
        Hide
        Ari Rabkin added a comment -

        I just committed this.

        Show
        Ari Rabkin added a comment - I just committed this.

          People

          • Assignee:
            Ari Rabkin
            Reporter:
            Ari Rabkin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development