[SPARK-19415] Improve the implicit type conversion between numeric type and string to avoid precesion loss

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Davies Liu
• Created: 31/Jan/17 22:47
• Updated: 31/Jan/17 23:20
• Resolved: 31/Jan/17 23:20

Description

Currently, Spark SQL will convert both numeric type and string into DoubleType, if the two children of a expression does not match (for example, comparing a LongType again StringType), this will cause precision loss in some cases.

Some database does better job one this (for example, SQL Server [1]), we should follow that.

[1] https://msdn.microsoft.com/en-us/library/ms191530.aspx

Issue Links

• is duplicated by: SPARK-17913 Filter/join expressions can return incorrect results when comparing strings to longs