Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-2064

Compress Hook Messages Posted to Kafka Atlas Topic

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 0.8-incubating
    • None
    • atlas-intg
    • None

    Description

      Background
      Messages posted by hooks to Atlas Kafka topics are JSON format.

      Kafka imposes a 1MB limit on the message size.

      Occasionally, depending on operations performed, this threshold is reached. This results in messages being dropped.

      The entities are thus not reflected in Atlas.

      Solution
      Applying compression to these message will alleviate the problem.

      Attachments

        Issue Links

        duplicates

        Improvement - An improvement or enhancement to an existing feature or task. ATLAS-2075 Support Messages from Hooks of Arbitrarily Large Size

        • Major - Major loss of function.
        • Resolved
        links to

        Web Link Code Review

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            amestry Ashutosh Mestry
            amestry Ashutosh Mestry
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment