[KAFKA-16456] Can't stop kafka debug logs

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 3.6.0
• Component/s: logging
• Assignee: Unassigned
• Reporter: Rajan Choudhary
• Created: 02/Apr/24 08:07
• Updated: 09/Apr/24 07:31
• Resolved: 09/Apr/24 07:31

Description

I am getting kafka debug logs, which are flooding our logs. Sample below

 

09:50:38.073 [kafka-producer-network-thread | maximo-mp] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=maximo-mp, transactionalId=sqout-3664816744674374805414] Received API_VERSIONS response from node 5 for request with header RequestHeader(apiKey=API_VERSIONS, apiVersion=3, clientId=maximo-mp, correlationId=8): ApiVersionsResponseData(errorCode=0, apiKeys=[ApiVersion(apiKey=0, minVersion=0, maxVersion=9), ApiVersion(apiKey=1, minVersion=0, maxVersion=13), ApiVersion(apiKey=2, minVersion=0, maxVersion=7), ApiVersion(apiKey=3, minVersion=0, maxVersion=12), ApiVersion(apiKey=4, minVersion=0, maxVersion=5), ApiVersion(apiKey=5, minVersion=0, maxVersion=3), ApiVersion(apiKey=6, minVersion=0, maxVersion=7), ApiVersion(apiKey=7, minVersion=0, maxVersion=3), ApiVersion(apiKey=8, minVersion=0, maxVersion=8), ApiVersion(apiKey=9, minVersion=0, maxVersion=8), ApiVersion(apiKey=10, minVersion=0, maxVersion=4), ApiVersion(apiKey=11, minVersion=0, maxVersion=7), ApiVersion(apiKey=12, minVersion=0, m...

09:50:38.073 [kafka-producer-network-thread | maximo-mp] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=maximo-mp, transactionalId=sqout-3664816744674374805414] Node 5 has finalized features epoch: 1, finalized features: [], supported features: [], API versions: (Produce(0): 0 to 9 [usable: 9], Fetch(1): 0 to 13 [usable: 12], ListOffsets(2): 0 to 7 [usable: 6], Metadata(3): 0 to 12 [usable: 11], LeaderAndIsr(4): 0 to 5 [usable: 5], StopReplica(5): 0 to 3 [usable: 3], UpdateMetadata(6): 0 to 7 [usable: 7], ControlledShutdown(7): 0 to 3 [usable: 3], OffsetCommit(8): 0 to 8 [usable: 8], OffsetFetch(9): 0 to 8 [usable: 7], FindCoordinator(10): 0 to 4 [usable: 3], JoinGroup(11): 0 to 7 [usable: 7], Heartbeat(12): 0 to 4 [usable: 4], LeaveGroup(13): 0 to 4 [usable: 4], SyncGroup(14): 0 to 5 [usable: 5], DescribeGroups(15): 0 to 5 [usable: 5], ListGroups(16): 0 to 4 [usable: 4], SaslHandshake(17): 0 to 1 [usable: 1], ApiVersions(18): 0 to 3 [usable: 3], CreateTopics(19): 0 to 7 [usable: 7], Del...

09:50:38.074 [kafka-producer-network-thread | maximo-mp] DEBUG org.apache.kafka.clients.producer.internals.TransactionManager - [Producer clientId=maximo-mp, transactionalId=sqout-3664816744674374805414] ProducerId of partition sqout-0 set to 43458621 with epoch 0. Reinitialize sequence at beginning.

09:50:38.074 [kafka-producer-network-thread | maximo-mp] DEBUG org.apache.kafka.clients.producer.internals.RecordAccumulator - [Producer clientId=maximo-mp, transactionalId=sqout-3664816744674374805414] Assigned producerId 43458621 and producerEpoch 0 to batch with base sequence 0 being sent to partition sqout-0

09:50:38.075 [kafka-producer-network-thread | maximo-mp] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=maximo-mp, transactionalId=sqout-3664816744674374805414] Sending PRODUCE request with header RequestHeader(apiKey=PRODUCE, apiVersion=9, clientId=maximo-mp, correlationId=9) and timeout 30000 to node 5: {acks=-1,timeout=30000,partitionSizes=[sqout-0=4181]}

09:50:38.095 [kafka-producer-network-thread | maximo-mp] DEBUG org.apache.kafka.clients.NetworkClient - [Producer clientId=maximo-mp, transactionalId=sqout-3664816744674374805414] Received PRODUCE response from node 5 for request with header RequestHeader(apiKey=PRODUCE, apiVersion=9, clientId=maximo-mp, correlationId=9): ProduceResponseData(responses=[TopicProduceResponse(name='sqout', partitionResponses=[PartitionProduceResponse(index=0, errorCode=0, baseOffset=796494, logAppendTimeMs=-1, logStartOffset=768203, recordErrors=[], errorMessage=null)])], throttleTimeMs=0)

09:50:38.095 [kafka-producer-network-thread | maximo-mp] DEBUG org.apache.kafka.clients.producer.internals.TransactionManager - [Producer clientId=maximo-mp, transactionalId=sqout-3664816744674374805414] ProducerId: 43458621; Set last ack'd sequence number for topic-partition sqout-0 to 0
09:50:38.147 [pool-7-thread-19] DEBUG org.apache.kafka.clients.producer.internals.TransactionManager - [Producer clientId=maximo-mp, transactionalId=sqout-3664816744674374805414] Transition from state IN_TRANSACTION to COMMITTING_TRANSACTION

09:50:38.147 [pool-7-thread-19] DEBUG org.apache.kafka.clients.producer.internals.TransactionManager - [Producer clientId=maximo-mp, transactionalId=sqout-3664816744674374805414] Enqueuing transactional request EndTxnRequestData(transactionalId='sqout-3664816744674374805414', producerId=43458621, producerEpoch=0, committed=true)
09:50:38