[SPARK-20063] Trigger without delay when falling behind

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: 2.1.0
• Component/s: Structured Streaming
• Target Version/s: 2.2.0
• Assignee: Unassigned
• Reporter: Michael Armbrust
• Created: 22/Mar/17 23:43
• Updated: 04/May/17 00:35
• Resolved: 04/May/17 00:35

Description

Today, when we miss a trigger interval we wait until the next one to fire. However, for real workloads this usually means that you fall further and further behind by sitting idle while waiting. We should revisit this decision.

Issue Links

• duplicates: SPARK-20209 Execute next trigger immediately if previous batch took longer than trigger interval