Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34651 Improve ZSTD support
  3. SPARK-33913

Upgrade Kafka to 2.8.0

Rank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.2.0
    • Build, DStreams
    • None

    Description

      This issue aims to upgrade Kafka client to 2.8.0.
      Note that Kafka 2.8.0 uses ZSTD JNI 1.4.9-1 like Apache Spark 3.2.0.

      RELEASE NOTE

      This will bring the latest client-side improvement and bug fixes like the following examples.

      • KAFKA-10631 ProducerFencedException is not Handled on Offest Commit
      • KAFKA-10134 High CPU issue during rebalance in Kafka consumer after upgrading to 2.5
      • KAFKA-12193 Re-resolve IPs when a client is disconnected
      • KAFKA-10090 Misleading warnings: The configuration was supplied but isn't a known config
      • KAFKA-9263 The new hw is added to incorrect log when  ReplicaAlterLogDirsThread is replacing log
      • KAFKA-10607 Ensure the error counts contains the NONE
      • KAFKA-10458 Need a way to update quota for TokenBucket registered with Sensor
      • KAFKA-10503 MockProducer doesn't throw ClassCastException when no partition for topic

      Attachments

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            dongjoon Dongjoon Hyun
            dengziming Deng Ziming
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment