Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8251

Consider different logging for purgeData

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: core
    • Labels:
      None

      Description

      As reported in this SO question https://stackoverflow.com/questions/55724259/kafka-streams-floods-kafka-logs/55738257#55738257 each time data is purged from a topic a log message is written:

      > {{[2019-04-17 09:06:16,541] INFO [Log partition=my-application-KSTREAM-AGGREGATE-STATE-STORE-0000000076-repartition-0, dir=/opt/kafka/data/logs] Incrementing log start offset to 316423 (kafka.log.Log) [2019-04-17 09:06:16,545] INFO [Log partition=my-application-KSTREAM-AGGREGATE-STATE-STORE-0000000033-repartition-2, dir=/opt/kafka/data/logs] Incrementing log start offset to 3394 (kafka.log.Log) }}

      While this is useful for regular log truncation, it seems less useful for explicit purge-data requests. Kafka Streams uses the purge data API on repartition topics aggressively and this can result to verbose logging.

      I don't know this part of the code, but maybe it's possible to distinguish both cases and only log on time/size based truncation at INFO level and reduce log level for purge-data request to TRACE?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mjsax Matthias J. Sax
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: