[GEARPUMP-342] Don't run DataSourceTask with WindowRunner unless necessary

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.8.4
• Fix Version/s: 0.8.5
• Assignee: Manu Zhang
• Reporter: Manu Zhang
• Created: 16/Aug/17 23:19
• Updated: 05/Jun/18 13:18
• Resolved: 05/Jun/18 13:18

Description

Currently, a windowed wordcount pipeline like source.flatMap.window.groupBy.count will actually be executed as source.globalWindow.flatMap.groupBy.window.count and globalWindow.flatMap run by a WindowRunner.

The globalWindow is unnecessary unless source is followed by an aggregation. Further, the globalWindow requires Watermark.MAX to trigger emits which is not obvious for users.

Issue Links

• duplicates: GEARPUMP-367 Unnecessary use of window processing