[HUDI-2816] Unify file listing method of Spark/Flink/Hive

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Labels: help-wanted query-eng sev:critical
• Assignee: XiaoyuGeng
• Reporter: Gary Li
• Created: 21/Nov/21 10:20
• Updated: 21/Jan/22 20:19
• Resolved: 21/Jan/22 20:19

Description

Currently, different engines are using different file listing methods, which might lead to inconsistent behavior. We need to unify these to deliver consistent behavior across different engines.

Issue Links

• duplicates: HUDI-3081 Revisiting Read Path Infra across Query Engines
• relates to: HUDI-2480 FileSlice after pending compaction-requested instant-time is ignored by MOR snapshot reader