Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10295

ConnectDistributedTest.test_bounce should wait for graceful stop

    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.3.1, 2.5.0, 2.4.1, 2.6.0
    • 2.3.2, 2.6.0, 2.4.2, 2.5.1, 2.7.0
    • KafkaConnect
    • None

    Description

      In ConnectDistributedTest.test_bounce, there are flakey failures that appear to follow this pattern:

      1. The test is parameterized for hard bounces, and with Incremental Cooperative Rebalancing enabled (does not appear for protocol=eager)
      2. A source task is on a worker that will experience a hard bounce
      3. The source task has written records which it has not yet committed in source offsets
      4. The worker is hard-bounced, and the source task is lost
      5. Incremental Cooperative Rebalance starts it's scheduled.rebalance.max.delay.ms delay before recovering the task
      6. The test ends, connectors and Connect are stopped
      7. The test verifies that the sink connector has only written records that have been committed by the source connector
      8. This verification fails because the source offsets are stale, and there are un-committed records in the topic, and the sink connector has written at least one of them.

      This can be addressed by ensuring that the test waits for the rebalance delay to expire, and for the lost task to recover and commit offsets past the progress it made before the bounce.

      Attachments

        Issue Links

          Activity

            People

              gharris1727 Greg Harris
              gharris1727 Greg Harris
              Randall Hauch Randall Hauch
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: