[SPARK-47625] Addition of Indeterminate Collation Support

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Mihailo Milosevic
• Created: 28/Mar/24 09:24
• Updated: 11/Apr/24 10:29

Description

INDETERMINATE_COLLATION should only be thrown on comparison operations and memory storing of data, and we should be able to combine different implicit collations for certain operations like concat and possible others in the future.
This is why we have to add another predefined collation id named INDETERMINATE_COLLATION_ID which means that the result is a combination of conflicting non-default implicit collations. Right now it would an id of -1 so it fail if it ever goes to the CollatorFactory.

Issue Links

• links to: GitHub Pull Request #46004