[HUDI-1420] HoodieTableMetaClient.getMarkerFolderPath works incorrectly on windows client with hdfs server for wrong file seperator

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 0.8.0
• Component/s: Common Core
• Labels: core-flow-ds pull-request-available sev:critical
• Assignee: 谢波
• Reporter: Sean Zhang
• Created: 27/Nov/20 08:25
• Updated: 16/Dec/21 01:31

Description

The method assembly market dir with File.separator, here File references java.io.File. But when a hudi app run on a windows system (for example, spark local app on windows jdk), the constant is "\", but if the hudi directory locates on HDFS, the seperator should be "/".

It cause the market is generated under directories like .hdfs://localhost:29820/HUDO_BASE/HUDI_TABLE_DIR\\.hoodie\\.temp
20201127161504, here HUDI_TABLE_DIR\\.hoodie\\.temp
20201127161504 is a whole directory name. And hudi app report some out market directory exception.

Issue Links

• Blocked: HUDI-2451 HoodieTableMetaClient The file separator from Window to HDFS is faulty
• links to: GitHub Pull Request #2525
• links to: GitHub Pull Request #2526