[SPARK-28433] Incorrect assertion in scala test for aarch64 platform

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Assignee: huangtianhua
• Reporter: huangtianhua
• Shepherd: Sean R. Owen
• Created: 18/Jul/19 03:38
• Updated: 19/Nov/19 01:32
• Resolved: 19/Jul/19 23:59

Description

We ran unit tests of spark on aarch64 server, here are two sql scala tests failed:

• SPARK-26021: NaN and -0.0 in grouping expressions *** FAILED ***
  2143289344 equaled 2143289344 (DataFrameAggregateSuite.scala:732)
• NaN and -0.0 in window partition keys *** FAILED ***
  2143289344 equaled 2143289344 (DataFrameWindowFunctionsSuite.scala:704)

we found the values of floatToRawIntBits(0.0f / 0.0f) and floatToRawIntBits(Float.NaN) on aarch64 are same(2143289344), first we thought it's something about jdk or scala, but after discuss with jdk-dev and scala community see https://users.scala-lang.org/t/the-value-of-floattorawintbits-0-0f-0-0f-is-different-on-x86-64-and-aarch64-platforms/4845 , we believe the value depends on the architecture.

Issue Links

• is part of: SPARK-29106 Add jenkins arm test for spark
• links to: GitHub Pull Request #25186