Details
-
Task
-
Status: In Progress
-
Blocker
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
FileIndex has to be providing its full-suite of capabilities in a generic way:
- Partition Pruning
- Column Stats Pruning
- Caching
- etc
To support partition-pruning as well as col-stats pruning in an engine-agnostic way, we'd have to implement our own Expression hierarchy supporting
- Conversion from engine-specific one to Hudi's hierarchy
- Being able to execute expressions