[SPARK-45986] Fix `pyspark.ml.torch.tests.test_distributor` in Python 3.11

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: PySpark
• Labels: pull-request-available
• Assignee: Hyukjin Kwon
• Reporter: Dongjoon Hyun
• Created: 18/Nov/23 23:05
• Updated: 20/Nov/23 01:36
• Resolved: 20/Nov/23 01:33

Description

https://github.com/apache/spark/actions/runs/6914662405/job/18812759511

======================================================================
FAIL [0.000s]: test_local_training_succeeds (pyspark.ml.torch.tests.test_distributor.TorchDistributorLocalUnitTests.test_local_training_succeeds) [subtest: 1]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/__w/spark/spark/python/pyspark/ml/torch/tests/test_distributor.py", line 384, in test_local_training_succeeds
    self.assertEqual(
AssertionError: '1' != '0'
- 1
+ 0


======================================================================
FAIL [0.142s]: test_local_training_succeeds (pyspark.ml.torch.tests.test_distributor.TorchDistributorLocalUnitTests.test_local_training_succeeds) [subtest: 2]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/__w/spark/spark/python/pyspark/ml/torch/tests/test_distributor.py", line 384, in test_local_training_succeeds
    self.assertEqual(
AssertionError: '1,2,0' != '0,1,2'
- 1,2,0
+ 0,1,2


======================================================================
FAIL [0.000s]: test_local_training_succeeds (pyspark.ml.torch.tests.test_distributor.TorchDistributorLocalUnitTestsII.test_local_training_succeeds) [subtest: 1]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/__w/spark/spark/python/pyspark/ml/torch/tests/test_distributor.py", line 384, in test_local_training_succeeds
    self.assertEqual(
AssertionError: '1' != '0'
- 1
+ 0


======================================================================
FAIL [0.139s]: test_local_training_succeeds (pyspark.ml.torch.tests.test_distributor.TorchDistributorLocalUnitTestsII.test_local_training_succeeds) [subtest: 2]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/__w/spark/spark/python/pyspark/ml/torch/tests/test_distributor.py", line 384, in test_local_training_succeeds
    self.assertEqual(
AssertionError: '1,2,0' != '0,1,2'
- 1,2,0
+ 0,1,2


----------------------------------------------------------------------
Ran 23 tests in 166.741s

FAILED (failures=4)

Issue Links

• links to: GitHub Pull Request #43892