[BEAM-10493] Unbounded sources unable to recover from checkpointMark when withMaxReadTime() is used

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Affects Version/s: 2.22.0
• Component/s: sdk-java-core
• Labels: Clarified
• Assignee: Unassigned
• Reporter: Mani Kolbe
• Created: 15/Jul/20 14:12
• Updated: 04/Jun/22 17:21

Description

Observed on v2.22.0 

When withMaxReadTime() is used, Beam creates an BoundedReadFromUnboundedSource [1].  The ReadFn<T> class in BoundedReadFromUnboundedSource which is responsible for reading records from source. You can see this class doesnt verify if there is a recoverable checkpoint exist. Instead it always creates Reader with checkpointMark set as null [2]. 

 

Reference:

[1] https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/Read.java#L205

[2]https://github.com/apache/beam/blob/v2.22.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/BoundedReadFromUnboundedSource.java#L193