[SPARK-18772] Unnecessary conversion try for special floats in JSON

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.2, 2.2.0
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Hyukjin Kwon
• Reporter: Nathan Howell
• Created: 07/Dec/16 23:28
• Updated: 12/Dec/22 18:10
• Resolved: 13/May/17 13:02

Description

It looks we can avoid some cases for unnecessary conversion try in special floats in JSON.

scala> import org.apache.spark.sql.types._
import org.apache.spark.sql.types._

scala> spark.read.schema(StructType(Seq(StructField("a", DoubleType)))).option("mode", "FAILFAST").json(Seq("""{"a": "nan"}""").toDS).show()
17/05/12 11:30:41 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 2)
java.lang.NumberFormatException: For input string: "nan"
...

Issue Links

• links to: [Github] Pull Request #16199 (NathanHowell)
• links to: [Github] Pull Request #17956 (HyukjinKwon)