[FLINK-23725] HadoopFsCommitter, file rename failure

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.11.1, 1.12.1
• Component/s: Connectors / FileSystem, Connectors / Hadoop Compatibility, FileSystems
• Assignee: Unassigned
• Reporter: todd
• Created: 11/Aug/21 11:59
• Updated: 13/Jan/22 12:07

Description

When the HDFS file is written, if the part file exists, only false will be returned if the duplicate name fails.Whether to throw an exception that already exists in the part, or print related logs.

 

```

org.apache.flink.runtime.fs.hdfs.HadoopRecoverableFsDataOutputStream.HadoopFsCommitter#commit

public void commit() throws IOException {
final Path src = recoverable.tempFile();
final Path dest = recoverable.targetFile();
final long expectedLength = recoverable.offset();

try

{      //always   return false or ture     fs.rename(src, dest); }

catch (IOException e)

{ throw new IOException( "Committing file by rename failed: " + src + " to " + dest, e); }

}