Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1590

Binarize trace level request logging along with debug level text logging

Agile BoardAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      With trace level logging, the request handling logs can grow very fast depending on the client behavior (e.g. consumer with 0 maxWait and hence keep sending fetch requests). Previously we have changed it to debug level which only provides a summary of the requests, omitting request details. However this does not work perfectly since summaries are not sufficient for trouble-shooting, and turning on trace level upon issues will be too late.

      The proposed solution here, is to default to debug level logging with trace level logging printed as binary format at the same time. The generated binary files can then be further compressed / rolled out. When needed, we will then decompress / parse the trace logs into texts.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            abhioncbr Abhishek Sharma
            guozhang Guozhang Wang

            Dates

              Created:
              Updated:

              Slack

                Issue deployment