Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10295

ConnectDistributedTest.test_bounce should wait for graceful stop

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.3.1, 2.5.0, 2.4.1, 2.6.0
    • Fix Version/s: 2.3.2, 2.6.0, 2.4.2, 2.5.1, 2.7.0
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      In ConnectDistributedTest.test_bounce, there are flakey failures that appear to follow this pattern:

      1. The test is parameterized for hard bounces, and with Incremental Cooperative Rebalancing enabled (does not appear for protocol=eager)
      2. A source task is on a worker that will experience a hard bounce
      3. The source task has written records which it has not yet committed in source offsets
      4. The worker is hard-bounced, and the source task is lost
      5. Incremental Cooperative Rebalance starts it's scheduled.rebalance.max.delay.ms delay before recovering the task
      6. The test ends, connectors and Connect are stopped
      7. The test verifies that the sink connector has only written records that have been committed by the source connector
      8. This verification fails because the source offsets are stale, and there are un-committed records in the topic, and the sink connector has written at least one of them.

      This can be addressed by ensuring that the test waits for the rebalance delay to expire, and for the lost task to recover and commit offsets past the progress it made before the bounce.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gharris1727 Greg Harris
                Reporter:
                gharris1727 Greg Harris
                Reviewer:
                Randall Hauch
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: