[KYLIN-4180] Prevent abnormal CPU usage by limiting flat filters length

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: v2.4.0, v2.5.0, v2.6.0
• Fix Version/s: v3.0.0, v2.6.5
• Component/s: Metadata
• Assignee: Temple Zhou
• Reporter: Temple Zhou
• Created: 26/Sep/19 06:53
• Updated: 21/Jan/20 07:48
• Resolved: 18/Nov/19 06:14

Description

strong textThe following SQL will lead to the list returned by org.apache.kylin.metadata.filter.TupleFilter#cartesianProduct become very large, and the process to producing the List will use lots of CPU.

select * from temp.test_filter
where ((a >= 0 and a <= 1)
or    (a >= 1 and a <= 2)
or    (a >= 2 and a <= 3)
...
or    (a >= 300 and a <= 301)
) and ((b >= 0 and b <= 1)
or    (b >= 1 and b <= 2)
or    (b >= 2 and b <= 3)
...
or    (b >= 300 and b <= 301)
)

Issue Links

• is caused by: KYLIN-3797 Too many or filters may break Kylin server when flatting filter
• links to: GitHub Pull Request #861