[HUDI-5008] Avoid unset HoodieROTablePathFilter in IncrementalRelation

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.12.2
• Component/s: incremental-query, spark
• Labels: pull-request-available
• Assignee: Hui An
• Reporter: Hui An
• Created: 11/Oct/22 07:09
• Updated: 14/Dec/22 03:30
• Resolved: 24/Nov/22 07:36

Description

If users create an incrementalRelation while join another existing hive hudi table, as pathFilter is unset inside incrementalRelation, all files under hive hudi table will be selected.

Now HoodieROTablePathFilter can accept as.of.instant to do the time travel, so instead we pass as.of.instant to the dataframe(not change spark hadoop conf globally) to avoid this issue.

Issue Links

• links to: GitHub Pull Request #6921