Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4036

Unsafe dictionary access in VerifiableConsumer service

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: system tests
    • Labels:
      None

      Description

      Found this in a recent run. Looks like not all changes to the dictionary are guarded with the lock.

      test_id:    2016-08-12--001.kafkatest.tests.client.consumer_test.OffsetValidationTest.test_consumer_failure.clean_shutdown=True.enable_autocommit=False
      status:     FAIL
      run time:   1 minute 32.957 seconds
      
      
          dictionary changed size during iteration
      Traceback (most recent call last):
        File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder/kafka/venv/local/lib/python2.7/site-packages/ducktape/tests/runner.py", line 106, in run_all_tests
          data = self.run_single_test()
        File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder/kafka/venv/local/lib/python2.7/site-packages/ducktape/tests/runner.py", line 162, in run_single_test
          return self.current_test_context.function(self.current_test)
        File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder/kafka/venv/local/lib/python2.7/site-packages/ducktape/mark/_mark.py", line 331, in wrapper
          return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
        File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder/kafka/tests/kafkatest/tests/client/consumer_test.py", line 163, in test_consumer_failure
          self.await_all_members(consumer)
        File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder/kafka/tests/kafkatest/tests/verifiable_consumer_test.py", line 83, in await_all_members
          self.await_members(consumer, self.num_consumers)
        File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder/kafka/tests/kafkatest/tests/verifiable_consumer_test.py", line 80, in await_members
          err_msg="Consumers failed to join in a reasonable amount of time")
        File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder/kafka/venv/local/lib/python2.7/site-packages/ducktape/utils/util.py", line 31, in wait_until
          if condition():
        File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder/kafka/tests/kafkatest/tests/verifiable_consumer_test.py", line 78, in <lambda>
          wait_until(lambda: len(consumer.joined_nodes()) == num_consumers,
        File "/var/lib/jenkins/workspace/system-test-kafka-branch-builder/kafka/tests/kafkatest/services/verifiable_consumer.py", line 317, in joined_nodes
          return [handler.node for handler in self.event_handlers.itervalues()
      RuntimeError: dictionary changed size during iteration
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                hachikuji Jason Gustafson
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: