[HDFS-17268] when SocketTimeoutException happen, overwrite mode can delete old data, and make file empty

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Resolved
• Affects Version/s: 3.2.2
• Assignee: Unassigned
• Reporter: katty he
• Created: 01/Dec/23 07:12
• Updated: 04/Dec/23 05:43
• Resolved: 04/Dec/23 05:43

Description

recently, i use fs.create(path, true/* createOrOverwrite */) to write data into parquet file a, but when SocketTimeoutException happend, such as " org.apache.hadoop.io.retry.RetryInvocationHandler [] - java.net.SocketTimeoutException: Call From xxx to namenodexxx:8888 failed on socket timeout exception: java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/node:33416 remote=namenode:8888]; For more details see: http://wiki.apache.org/hadoop/SocketTimeout, while invoking ClientNamenodeProtocolTranslatorPB.create over namenode:8888. Trying to failover immediately." then i found the size of file a is zero, and read with error "file a is not a parquet file", and there were two create calls from two different routers in hdfs audit log, so i think overwrite is not safe in some situation

Issue Links

• is fixed by: HDFS-15079 RBF: Client maybe get an unexpected result with network anomaly