[FLINK-15533] Writing DataStream as text file fails due to output path already exists

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.10.0
• Component/s: Client / Job Submission
• Labels: pull-request-available
• Assignee: Kostas Kloudas
• Reporter: Rui Li
• Created: 09/Jan/20 07:27
• Updated: 14/Jan/20 10:12
• Resolved: 14/Jan/20 10:12

Description

The following program reproduces the issue.

    Configuration configuration = GlobalConfiguration.loadConfiguration();
    configuration.set(DeploymentOptions.TARGET, RemoteExecutor.NAME);
    StreamExecutionEnvironment streamEnv = new StreamExecutionEnvironment(configuration);

    DataStream dataStream = streamEnv.fromCollection(Arrays.asList(1,2,3));
    dataStream.writeAsText("hdfs://localhost:8020/tmp/output");

    streamEnv.execute();

The job will fail with the follow error, even though the output path doesn't exist before job submission:

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.FileAlreadyExistsException): /tmp/output already exists as a directory

Issue Links

• relates to: FLINK-15511 export org.apache.flink.table.api.TableException when flink 1.10 connect hive
• links to: GitHub Pull Request #10835