Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2927

System tests: reduce storage footprint of collected logs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Looking at recent night test runs (testing.confluent.io/kafka), the storage requirements for log output from the various services has increased significantly, up to 7-10G for a single test run, up from hundreds of MB

      Current breakdown:
      23M Benchmark
      3.2M ClientCompatibilityTest
      613M ConnectDistributedTest
      1.1M ConnectRestApiTest
      1.5M ConnectStandaloneFileTest
      2.0M ConsoleConsumerTest
      440K KafkaVersionTest
      744K Log4jAppenderTest
      49M QuotaTest
      3.0G ReplicationTest
      1.2G TestMirrorMakerService
      185M TestUpgrade
      372K TestVerifiableProducer
      2.3G VerifiableConsumerTest

      The biggest contributors in these test suites:

      ReplicationTest:
      verifiable_producer.log (currently TRACE level)

      VerifiableConsumerTest:
      kafka server.log

      TestMirrorMakerService:
      verifiable_producer.log

      ConnectDistributedTest:
      kafka server.log

      The worst offenders are therefore
      verifiable_producer.log which is logging at TRACE level, and kafka server.log which is logging at debug level

      One solution is to:
      1) Update the log4j configs to log separately to both an INFO level file, and another file for DEBUG at least for the worst offenders.

      2) Don't collect these DEBUG (and below) logs by default; only mark for collection during failure

        Attachments

          Activity

            People

            • Assignee:
              granders Geoff Anderson
              Reporter:
              granders Geoff Anderson
              Reviewer:
              Ewen Cheslack-Postava
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: