[FLINK-3964] Job submission times out with recursive.file.enumeration

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Component/s: API / DataSet
• Assignee: Maximilian Michels
• Reporter: Juho Autio
• Created: 24/May/16 11:04
• Updated: 28/Feb/19 13:46
• Resolved: 27/Feb/19 19:28

Description

When using "recursive.file.enumeration" with a big enough folder structure to list, flink batch job fails right at the beginning because of a timeout.

Problem details

We get this error: Communication with JobManager failed: Job submission to the JobManager timed out.

The code we have is basically this:

val env = ExecutionEnvironment.getExecutionEnvironment

val parameters = new Configuration

// set the recursive enumeration parameter
parameters.setBoolean("recursive.file.enumeration", true)

val parameter = ParameterTool.fromArgs(args)

val input_data_path : String = parameter.get("input_data_path", null )

val data : DataSet[(Text,Text)] = env.readSequenceFile(classOf[Text], classOf[Text], input_data_path)
.withParameters(parameters)

data.first(10).print

If we set input_data_path parameter to s3n://bucket/path/date=*/ it times out. If we use a more restrictive pattern like s3n://bucket/path/date=20160523/, it doesn't time out.

To me it seems that time taken to list files shouldn't cause any timeouts on job submission level.

For us this was "fixed" by adding akka.client.timeout: 600 s in flink-conf.yaml, but I wonder if the timeout would still occur if we have even more files to list?

---

P.S. Is there any way to set akka.client.timeout when calling bin/flink run instead of editing flink-conf.yaml. I tried to add it as a -yD flag but couldn't get it working.

Issue Links

• links to: GitHub Pull Request #2168