Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-40294

Repeat calls to `PartitionIterator.hasNext` can timeout

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 3.3.0
    • None
    • SQL
    • None

    Description

      Repeat calls to PartitionIterator.hasNext where both calls return false can result in timeouts. For example, KafkaBatchPartitionReader.next(), which calls consumer.get (which can potentially timeout with repeat calls), is called by PartitionIterator.hasNext. Thus, repeat calls to PartitionIterator.hasNext by its parent could timeout.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              richardc-db Richard Chen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: