Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6486

TimeWindows causes unordered calls to windowed aggregation functions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.0.0
    • 2.0.0
    • streams
    • None

    Description

      This is not a real bug but it causes some weird behaviour, at least in my opinion.

      The TimeWindows has a method called windowsFor() that uses and returns an HashMap:

          @Override

          public Map<Long, TimeWindow> windowsFor(final long timestamp) {

              long windowStart = (Math.max(0, timestamp - sizeMs + advanceMs) / advanceMs) * advanceMs;

              final Map<Long, TimeWindow> windows = new HashMap<>();

              ....

      the HashMap does not preserve the order of insertion and this ends up later in calls to any streams windowed aggregation functions that are not ordered by window time as I would expect.

      A simple solution is to replace the HashMap with a LinkedHashMap and that's what I did.

      Anyway replacing it directly in your code can save hours of debugging to understand what's happening.

      Thank you 

       

      Attachments

        1. KAFKA-6486.patch
          1 kB
          Asutosh Pandya

        Issue Links

          Activity

            People

              asutosh_pandya Asutosh Pandya
              vale68 Valentino Proietti
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: