[BEAM-3907] Clarify how watermark is estimated for watchForNewFiles() transforms

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.6.0
• Component/s: sdk-java-core
• Labels: Done
• Assignee: Eugene Kirpichov
• Reporter: Chamikara Madhusanka Jayalath
• Created: 21/Mar/18 23:55
• Updated: 06/Oct/20 21:51
• Resolved: 02/Jul/18 23:22

Description

For example https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/AvroIO.java#L488

It's not clear how the watermark will be estimated/incremented when using these transforms. 

Other source implementations seems to be describing this. For example: https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L89