Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5375

Transactions: Concurrent transactional consumer loses messages when there are broker bounces

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0.0
    • Component/s: None
    • Labels:

      Description

      With the system test modifications in KAFKA-5366, the concurrent reader almost always consumes a fraction of the expected messages when there are broker bounces. A consumer running without concurrent writes consumes all the messages from the topic in question in the same test.

      ================================================================================
      test_id:    kafkatest.tests.core.transactions_test.TransactionsTest.test_transactions.failure_mode=clean_bounce.bounce_target=brokers
      status:     FAIL
      run time:   1 minute 59.169 seconds
      
      
          Input and concurrently consumed output message sets are not equal. Num input messages: 20000. Num concurrently_consumed_messages: 0
      Traceback (most recent call last):
        File "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py", line 123, in run
          data = self.run_test()
        File "/usr/local/lib/python2.7/dist-packages/ducktape/tests/runner_client.py", line 176, in run_test
          return self.test_context.function(self.test)
        File "/usr/local/lib/python2.7/dist-packages/ducktape/mark/_mark.py", line 321, in wrapper
          return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
        File "/opt/kafka-dev/tests/kafkatest/tests/core/transactions_test.py", line 238, in test_transactions
          (len(input_message_set), len(concurrently_consumed_message_set))
      AssertionError: Input and concurrently consumed output message sets are not equal. Num input messages: 20000. Num concurrently_consumed_messages: 0
      

        Attachments

        1. KAFKA-5375.tar.gz
          14.84 MB
          Apurva Mehta

          Activity

            People

            • Assignee:
              apurva Apurva Mehta
              Reporter:
              apurva Apurva Mehta
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: