Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28367

Kafka connector infinite wait because metadata never updated

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.1.3, 2.2.3, 2.3.3, 2.4.3, 3.0.0, 3.1.0
    • Fix Version/s: 3.1.0
    • Component/s: Structured Streaming
    • Labels:
      None

      Description

      Spark uses an old and deprecated API named poll(long) which never returns and stays in live lock if metadata is not updated (for instance when broker disappears at consumer creation).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gsomogyi Gabor Somogyi
                Reporter:
                gsomogyi Gabor Somogyi
              • Votes:
                2 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: