Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6486

TimeWindows causes unordered calls to windowed aggregation functions

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 2.0.0
    • Component/s: streams
    • Labels:
      None

      Description

      This is not a real bug but it causes some weird behaviour, at least in my opinion.

      The TimeWindows has a method called windowsFor() that uses and returns an HashMap:

          @Override

          public Map<Long, TimeWindow> windowsFor(final long timestamp) {

              long windowStart = (Math.max(0, timestamp - sizeMs + advanceMs) / advanceMs) * advanceMs;

              final Map<Long, TimeWindow> windows = new HashMap<>();

              ....

      the HashMap does not preserve the order of insertion and this ends up later in calls to any streams windowed aggregation functions that are not ordered by window time as I would expect.

      A simple solution is to replace the HashMap with a LinkedHashMap and that's what I did.

      Anyway replacing it directly in your code can save hours of debugging to understand what's happening.

      Thank you 

       

        Attachments

        1. KAFKA-6486.patch
          1 kB
          Asutosh Pandya

          Issue Links

            Activity

              People

              • Assignee:
                asutosh_pandya Asutosh Pandya
                Reporter:
                vale68 Valentino Proietti
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: