[SPARK-24834] Utils#nanSafeCompare{Double,Float} functions do not differ from normal java double/float comparison

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 2.3.2
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Benjamin Duffield
• Created: 17/Jul/18 14:23
• Updated: 12/Nov/18 18:10
• Resolved: 20/Aug/18 20:02

Description

Utils.scala contains two functions `nanSafeCompareDoubles` and `nanSafeCompareFloats` which purport to have special handling of NaN values in comparisons.

The handling in these functions do not appear to differ from java.lang.Double.compare and java.lang.Float.compare - they seem to produce identical output to the built-in java comparison functions.

I think it's clearer to not have these special Utils functions, and instead just use the standard java comparison functions.

Issue Links

• links to: [Github] Pull Request #21794 (bavardage)