[FLINK-9545] Support read a file multiple times in Flink DataStream

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.6.0
• Fix Version/s: 1.6.0
• Component/s: API / DataStream
• Assignee: Bowen Li
• Reporter: Bowen Li
• Created: 06/Jun/18 23:20
• Updated: 15/Jun/18 17:13
• Resolved: 15/Jun/18 17:13

Description

Motivation: We have the requirements to read a bunch files, each file to read multiple times, to feed our streams

Specifically we need StreamExecutionEnvironment.readFile/readTextFile to be able to read a file for a specified N times, but currently it only supports reading file once.

We've implemented this internally. Would be good to get it back to the community version. This jira is to add support for the feature.

Plan:

add a new processing mode as PROCESSING_N_TIMES, and add additional parameter numTimes for StreamExecutionEnvironment.readFile/readTextFile

Issue Links

• links to: GitHub Pull Request #6130