[BEAM-3161] Cannot output with timestamp XXXX

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.3.0
• Component/s: dsl-sql
• Assignee: Mingmin Xu
• Reporter: Mingmin Xu
• Created: 08/Nov/17 21:41
• Updated: 24/Jul/20 20:17
• Resolved: 13/Nov/17 08:13

Description

BeamSQL throws exception when running query

select siteId, count(*), TUMBLE_START(unix_timestamp_to_date(eventTimestamp), INTERVAL '1' HOUR) from RHEOS_SOJEVENT_TOTAL GROUP BY siteId, TUMBLE(unix_timestamp_to_date(eventTimestamp), INTERVAL '1' HOUR)

Exception details:

Caused by: java.lang.IllegalArgumentException: Cannot output with timestamp 2017-11-08T21:37:46.000Z. Output timestamps must be no earlier than the timestamp of the current input (2017-11-08T21:37:49.322Z) minus the allowed skew (0 milliseconds). See the DoFn#getAllowedTimestampSkew() Javadoc for details on changing the allowed skew.
	at org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.checkTimestamp(SimpleDoFnRunner.java:463)
	at org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.outputWithTimestamp(SimpleDoFnRunner.java:429)
	at org.apache.Caused by: java.lang.IllegalArgumentException: Cannot output with timestamp 2017-11-08T21:37:46.000Z. Output timestamps must be no earlier than the timestamp of the current input (2017-11-08T21:37:49.322Z) minus the allowed skew (0 milliseconds). See the DoFn#getAllowedTimestampSkew() Javadoc for details on changing the allowed skew.
	at org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.checkTimestamp(SimpleDoFnRunner.java:463)
	at org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.outputWithTimestamp(SimpleDoFnRunner.java:429)
	at org.apache.beam.sdk.transforms.WithTimestamps$AddTimestampsDoFn.processElement(WithTimestamps.java:138)beam.sdk.transforms.WithTimestamps$AddTimestampsDoFn.processElement(WithTimestamps.java:138)

Issue Links

• links to: GitHub Pull Request #4101