[HUDI-2576] flink do checkpoint error because parquet file is missing

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.10.0
• Component/s: flink
• Assignee: Unassigned
• Reporter: liyuanzhao435
• Created: 19/Oct/21 01:54
• Updated: 27/Mar/22 16:58

Description

hudi:0.10.0, flink 1.13.1

some times when flink do checkpoint , error occurs,  the error shows a hudi parquet file is missing (says file not exists) : 

2021-10-19 09:20:03,796 INFO org.apache.hudi.io.storage.row.HoodieRowDataCreateHandle [] - start close hoodie row data
2021-10-19 09:20:03,800 WARN org.apache.hadoop.hdfs.DataStreamer [] - DataStreamer Exception
java.io.FileNotFoundException: File does not exist: /tmp/test_liyz2/aa/2ff301cc-8db2-478e-b707-e8f2327ba38f-0_0-1-4_20211019091917.parquet (inode 32234795) Holder DFSClient_NONMAPREDUCE_633610786_99 does not have any open files.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2815)

 

detail see  appendix