Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6551

Unbounded queues in WorkerSourceTask cause OutOfMemoryError

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      A Debezium user reported an OutOfMemoryError to us, with over 50,000 messages in the WorkerSourceTask#outstandingMessages map.

      This map is unbounded and I can't see any way of "rate limiting" which would control how many records are added to it. Growth can only indirectly be limited by reducing the offset flush interval, but as connectors can return large amounts of messages in single poll() calls that's not sufficient in all cases. Note the user reported this issue during snapshotting a database, i.e. a high number of records arrived in a very short period of time.

      To solve the problem I'd suggest to make this map backpressure-aware and thus prevent its indefinite growth, so that no further records will be polled from the connector until messages have been taken out of the map again.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                gunnar.morling Gunnar Morling
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: