[SPARK-37522] Fix MultilayerPerceptronClassifierTest.test_raw_and_probability_prediction

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.2.1, 3.3.0
• Component/s: PySpark, Tests
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 02/Dec/21 17:45
• Updated: 16/Dec/21 22:05
• Resolved: 02/Dec/21 18:32

Description

The failure happens on Java 17 native Apple Silicon version on Python 3.9/3.10.

$ java -version
openjdk version "17.0.1" 2021-10-19 LTS
OpenJDK Runtime Environment Zulu17.30+15-CA (build 17.0.1+12-LTS)
OpenJDK 64-Bit Server VM Zulu17.30+15-CA (build 17.0.1+12-LTS, mixed mode, sharing)

======================================================================
FAIL: test_raw_and_probability_prediction (pyspark.ml.tests.test_algorithms.MultilayerPerceptronClassifierTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/dongjoon/APACHE/spark-merge/python/pyspark/ml/tests/test_algorithms.py", line 104, in test_raw_and_probability_prediction
    self.assertTrue(np.allclose(result.rawPrediction, expected_rawPrediction, rtol=0.102))
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1 test in 7.385s

FAILED (failures=1)

Had test failures in pyspark.ml.tests.test_algorithms MultilayerPerceptronClassifierTest.test_raw_and_probability_prediction with python3; see logs.

Issue Links

• links to: [Github] Pull Request #34784 (dongjoon-hyun)
• links to: [Github] Pull Request #34784 (dongjoon-hyun)