[SPARK-47286] IN operator support

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Spark Core
• Target Version/s: 4.0.0
• Epic Link: Collation support
• Assignee: Unassigned
• Reporter: Aleksandar Tomic
• Created: 05/Mar/24 10:37
• Updated: 08/Apr/24 10:56
• Resolved: 08/Apr/24 10:56

Description

At this point following query works fine:
```
sql("select * from t1 where ucs_basic_lcase in ('aaa' collate 'ucs_basic_lcase', 'bbb' collate 'ucs_basic_lcase')").show()

```

But if we were to miss explicit collate or even mix collations:
```
sql("select * from t1 where ucs_basic_lcase in ('aaa' collate 'ucs_basic_lcase', 'bbb'").show()

```

Query would still run and return invalid results.