Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4779

Failure in kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.2.1, 0.11.0.0
    • Component/s: None
    • Labels:
      None

      Description

      This test failed on 01/29, on both trunk and 0.10.2, error message:

      The consumer has terminated, or timed out, on node ubuntu@worker3.
      Traceback (most recent call last):
        File "/var/lib/jenkins/workspace/system-test-kafka-0.10.2/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/tests/runner_client.py", line 123, in run
          data = self.run_test()
        File "/var/lib/jenkins/workspace/system-test-kafka-0.10.2/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/tests/runner_client.py", line 176, in run_test
          return self.test_context.function(self.test)
        File "/var/lib/jenkins/workspace/system-test-kafka-0.10.2/kafka/venv/local/lib/python2.7/site-packages/ducktape-0.6.0-py2.7.egg/ducktape/mark/_mark.py", line 321, in wrapper
          return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
        File "/var/lib/jenkins/workspace/system-test-kafka-0.10.2/kafka/tests/kafkatest/tests/core/security_rolling_upgrade_test.py", line 148, in test_rolling_upgrade_phase_two
          self.run_produce_consume_validate(self.roll_in_secured_settings, client_protocol, broker_protocol)
        File "/var/lib/jenkins/workspace/system-test-kafka-0.10.2/kafka/tests/kafkatest/tests/produce_consume_validate.py", line 100, in run_produce_consume_validate
          self.stop_producer_and_consumer()
        File "/var/lib/jenkins/workspace/system-test-kafka-0.10.2/kafka/tests/kafkatest/tests/produce_consume_validate.py", line 87, in stop_producer_and_consumer
          self.check_alive()
        File "/var/lib/jenkins/workspace/system-test-kafka-0.10.2/kafka/tests/kafkatest/tests/produce_consume_validate.py", line 79, in check_alive
          raise Exception(msg)
      Exception: The consumer has terminated, or timed out, on node ubuntu@worker3.
      

      Looks like the console consumer times out:

      [2017-01-30 04:56:00,972] ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
      kafka.consumer.ConsumerTimeoutException
              at kafka.consumer.NewShinyConsumer.receive(BaseConsumer.scala:90)
              at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:120)
              at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:75)
              at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:50)
              at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
      

      A bunch of these security_rolling_upgrade tests failed, and in all cases, the producer produced ~15k messages, of which ~7k were acked, and the consumer only got around ~2600 before timing out.

      There are a lot of messages like the following for different request types on the producer and consumer:

      [2017-01-30 05:13:35,954] WARN Received unknown topic or partition error in produce request on partition test_topic-0. The topic/partition may not exist or the user may not have Describe access to it (org.apache.kafka.clients.producer.internals.Sender)
      

        Attachments

          Activity

            People

            • Assignee:
              rsivaram Rajini Sivaram
              Reporter:
              apurva Apurva Mehta
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: