[FLINK-10474] Don't translate IN with Literals to JOIN with VALUES for streaming queries

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.1, 1.7.0
• Fix Version/s: 1.7.0
• Component/s: Table SQL / API
• Labels: pull-request-available
• Assignee: Hequn Cheng
• Reporter: Fabian Hueske
• Created: 01/Oct/18 14:25
• Updated: 19/Sep/19 19:18
• Resolved: 16/Oct/18 09:26

Description

IN predicates with literals are translated to JOIN with VALUES if the number of elements in the IN clause exceeds a certain threshold. This should not be done, because a streaming join is very heavy and materializes both inputs (which is fine for the VALUES) input but not for the other.

There are two ways to solve this:

1. don't translate IN to a JOIN at all
2. translate it to a JOIN but have a special join strategy if one input is bound and final (non-updating)

Option 1. should be easy to do, option 2. requires much more effort.

Issue Links

• links to: GitHub Pull Request #6792