[SPARK-21774] The rule PromoteStrings cast string to a wrong data type

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Not A Problem
• Affects Version/s: 2.2.0
• Component/s: SQL
• Labels: correctness
• Assignee: Unassigned
• Reporter: StanZhai
• Created: 18/Aug/17 03:11
• Updated: 10/Sep/20 22:35
• Resolved: 10/Sep/20 22:35

Description

Data

create temporary view tb as select * from values
("0", 1),
("-0.1", 2),
("1", 3)
as grouping(a, b)

SQL:

select a, b from tb where a=0

The result which is wrong:

+----+---+
|   a|  b|
+----+---+
|   0|  1|
|-0.1|  2|
+----+---+

Logical Plan:

== Parsed Logical Plan ==
'Project ['a]
+- 'Filter ('a = 0)
   +- 'UnresolvedRelation `src`

== Analyzed Logical Plan ==
a: string
Project [a#8528]
+- Filter (cast(a#8528 as int) = 0)
   +- SubqueryAlias src
      +- Project [_1#8525 AS a#8528, _2#8526 AS b#8529]
         +- LocalRelation [_1#8525, _2#8526]

Issue Links

• relates to: SPARK-21646 Add new type coercion rules to compatible with Hive
• links to: [Github] Pull Request #18986 (stanzhai)
• links to: GitHub Pull Request #18986