[KAFKA-4526] Transient failure in ThrottlingTest.test_throttled_reassignment

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.2.0
• Labels: system-test-failure system-tests
• Assignee: Apurva Mehta
• Reporter: Ewen Cheslack-Postava
• Created: 12/Dec/16 21:33
• Updated: 19/Dec/16 23:42
• Resolved: 19/Dec/16 23:42

Description

This test is seeing transient failures sometimes

Module: kafkatest.tests.core.throttling_test
Class: ThrottlingTest
Method: test_throttled_reassignment
Arguments:
{
"bounce_brokers": false
}

This happens with both bounce_brokers = true and false. Fails with

AssertionError: 1646 acked message did not make it to the Consumer. They are: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19...plus 1626 more. Total Acked: 174799, Total Consumed: 173153. We validated that the first 1000 of these missing messages correctly made it into Kafka's data files. This suggests they were lost on their way to the consumer.

See http://confluent-kafka-system-test-results.s3-us-west-2.amazonaws.com/2016-12-12--001.1481535295--apache--trunk--62e043a/report.html for an example.

Note that there are a number of similar bug reports for different tests: https://issues.apache.org/jira/issues/?jql=text%20~%20%22acked%20message%20did%20not%20make%20it%20to%20the%20Consumer%22%20and%20project%20%3D%20Kafka I am wondering if we have a wrong ack setting somewhere that we should be specifying as acks=all but is only defaulting to 0?

It also seems interesting that the missing messages in these recent failures seem to always start at 0...

Issue Links

• links to: GitHub Pull Request #2278