[FLINK-2674] Rework windowing logic

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.10.0
• Fix Version/s: 1.0.0
• Component/s: API / DataStream
• Assignee: Unassigned
• Reporter: Stephan Ewen
• Created: 15/Sep/15 12:08
• Updated: 28/Feb/19 11:15
• Resolved: 04/Apr/16 14:53

Description

The windowing logic needs a major overhaul. This follows the design documents:

• https://cwiki.apache.org/confluence/display/FLINK/Time+and+Order+in+Streams
• https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=60624830

Specifically, the following shortcomings need to be addressed:

• Global parallel windows should be dropped
  -> for time, local windows are aligned and serve the same purpose
  -> there is currently no known robust and efficient parallel implementation of custom strategies

• Event time and out of order arrival needs to be supported

• Eviction of not accessed keys does not work. Non-accessed keys linger infinitely

• Performance is currently bad for time windows, due to a overly general implementation

• Resources are leaking, threads are not shut down

• Elements are stored multiple times (discretizers, window buffers)

• Finally, many implementations are buggy, produce wrong results