[SPARK-6624] Convert filters into CNF for data sources

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Auto Closed
• Component/s: SQL
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Reynold Xin
• Created: 31/Mar/15 04:25
• Updated: 01/Jun/20 08:34
• Resolved: 06/Jun/19 13:57

Description

We should turn filters into conjunctive normal form (CNF) before we pass them to data sources. Otherwise, filters are not very useful if there is a single filter with a bunch of ORs.

Note that we already try to do some of these in BooleanSimplification, but I think we should just formalize it to use CNF.

Issue Links

• relates to: SPARK-31705 Push more possible predicates through Join via CNF
• links to: [Github] Pull Request #8193 (yjshen)
• links to: [Github] Pull Request #8200 (yjshen)
• links to: [Github] Pull Request #10444 (liancheng)
• links to: [Github] Pull Request #15558 (viirya)