Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9259

suppress() for windowed-Serdes does not work with default serdes

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.1.0
    • Fix Version/s: None
    • Component/s: streams
    • Labels:

      Description

      The suppress() operator either inherits serdes from its upstream operator or falls back to default serdes from the config.

      If the upstream operator is an windowed aggregation, the window-aggregation operator wraps the user passed-in serde with a window-serde and pushed it into suppress() – however, if default serdes are used, the window-aggregation operator cannot push anything into suppress(). At runtime, it just creates a default serde and wraps it according. For this case, suppress() also falls back to default serdes; however, it does not wrap the serde and thus a ClassCastException is thrown when the serde is used later.

      suppress() is already aware if the upstream aggregation is time/session windowed or not and thus should use this information to wrap default serdes accordingly.

      The current workaround for windowed-suppress is to overwrite the default serde upstream to suppress(), such that suppress() inherits serdes and does not fall back to default serdes.

       

      For the search engines, when using Avro, this results in a ClassCastException:

      Caused by: java.lang.ClassCastException: class org.apache.kafka.streams.kstream.Windowed cannot be cast to class org.apache.avro.specific.SpecificRecord (org.apache.kafka.streams.kstream.Windowed and org.apache.avro.specific.SpecificRecord are in unnamed module of loader 'app') at io.confluent.kafka.streams.serdes.avro.SpecificAvroSerializer.serialize(SpecificAvroSerializer.java:38)

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              omanges Omkar Mestry
              Reporter:
              mjsax Matthias J. Sax

              Dates

              • Created:
                Updated:

                Issue deployment