[FLINK-1115] Spurious Test Failures due to failing file output streams

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.7.0-incubating
• Fix Version/s: 0.7.0-incubating
• Component/s: Runtime / Coordination
• Assignee: Stephan Ewen
• Reporter: Stephan Ewen
• Created: 23/Sep/14 00:07
• Updated: 23/Sep/14 21:54
• Resolved: 23/Sep/14 21:54

Description

Once in a while, a test fails with the stack trace blow, failing to open a file output stream. I am not fully sure what causes that, but I suspect a race between creating/clearing the output directory, and the file stream creation.

We could simply have one or two retries when the local file system' output stream creation fails with "FileNotFoundException"

---

23:39:57.272 [DataSink(Output) (1/4)] ERROR org.apache.flink.runtime.operators.DataSinkTask - Error in user code: /tmp/org.apache.flink.test.recordJobTests.TPCHQuery10ITCase-result.txt/1 (No such file or directory): DataSink(Output) (1/4)
java.io.FileNotFoundException: /tmp/org.apache.flink.test.recordJobTests.TPCHQuery10ITCase-result.txt/1 (No such file or directory)
at java.io.FileOutputStream.open(Native Method) ~[na:1.8.0_05]
at java.io.FileOutputStream.<init>(FileOutputStream.java:206) ~[na:1.8.0_05]
at java.io.FileOutputStream.<init>(FileOutputStream.java:156) ~[na:1.8.0_05]
at org.apache.flink.core.fs.local.LocalDataOutputStream.<init>(LocalDataOutputStream.java:50) ~[flink-core-0.7-incubating-SNAPSHOT.jar:0.7-incubating-SNAPSHOT]
at org.apache.flink.core.fs.local.LocalFileSystem.create(LocalFileSystem.java:247) ~[flink-core-0.7-incubating-SNAPSHOT.jar:0.7-incubating-SNAPSHOT]
at org.apache.flink.core.fs.local.LocalFileSystem.create(LocalFileSystem.java:254) ~[flink-core-0.7-incubating-SNAPSHOT.jar:0.7-incubating-SNAPSHOT]
at org.apache.flink.api.common.io.FileOutputFormat.open(FileOutputFormat.java:239) ~[flink-core-0.7-incubating-SNAPSHOT.jar:0.7-incubating-SNAPSHOT]
at org.apache.flink.runtime.operators.DataSinkTask.invoke(DataSinkTask.java:174) ~[flink-runtime-0.7-incubating-SNAPSHOT.jar:0.7-incubating-SNAPSHOT]
at org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:235) [flink-runtime-0.7-incubating-SNAPSHOT.jar:0.7-incubating-SNAPSHOT]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_05]