[SPARK-23624] Revise doc of method pushFilters

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.1
• Fix Version/s: 2.3.1, 2.4.0
• Component/s: SQL
• Assignee: Gengliang Wang
• Reporter: Gengliang Wang
• Created: 08/Mar/18 04:59
• Updated: 09/Mar/18 23:41
• Resolved: 09/Mar/18 23:41

Description

Revise doc of method pushFilters in SupportsPushDownFilters/SupportsPushDownCatalystFilters

In `FileSourceStrategy`,  except `partitionKeyFilters`(the references of which is subset of partition keys), all filters needs to be evaluated after scanning. Otherwise, Spark will get wrong result from data sources like Orc/Parquet.

 

Issue Links

• links to: [Github] Pull Request #20769 (gengliangwang)