[KYLIN-3797] Too many or filters may break Kylin server when flatting filter

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: v3.0.0-alpha, v2.6.4
• Component/s: Query Engine
• Assignee: Gang Ma
• Reporter: Gang Ma
• Created: 29/Jan/19 10:03
• Updated: 13/Oct/19 00:37
• Resolved: 13/Oct/19 00:34

Description

Kylin needs to convert filter into a flat format, like: OR(AND(f1,f2), AND(f3,f4,f5)…) , so that Kylin can build HBase scan/filter accordingly, but when there are too many or filters in query, like: AND(OR(f1,f2,...,f1000), OR(g1,g2,...,g1000), OR(h1,h2,...,h1000)), then the generated flat filter size will be 1000*1000*1000=1 billion, the flatting process will cause Kylin server OOM.

Issue Links

• causes: KYLIN-4180 Prevent abnormal CPU usage by limiting flat filters length
• links to: GitHub Pull Request #457