[SPARK-42089] Different result in nested lambda function

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: Connect, PySpark
• Assignee: Ruifeng Zheng
• Reporter: Ruifeng Zheng
• Created: 16/Jan/23 11:32
• Updated: 17/Jan/23 05:07
• Resolved: 17/Jan/23 05:07

Description

test_nested_higher_order_function

Traceback (most recent call last):
  File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/tests/test_functions.py", line 814, in test_nested_higher_order_function
    self.assertEquals(actual, expected)
AssertionError: Lists differ: [Row(n='a', l='a'), Row(n='b', l='b'), Row[124 chars]'c')] != [(1, 'a'), (1, 'b'), (1, 'c'), (2, 'a'), ([43 chars]'c')]

First differing element 0:
Row(n='a', l='a')
(1, 'a')

- [Row(n='a', l='a'),
-  Row(n='b', l='b'),
-  Row(n='c', l='c'),
-  Row(n='a', l='a'),
-  Row(n='b', l='b'),
-  Row(n='c', l='c'),
-  Row(n='a', l='a'),
-  Row(n='b', l='b'),
-  Row(n='c', l='c')]
+ [(1, 'a'),
+  (1, 'b'),
+  (1, 'c'),
+  (2, 'a'),
+  (2, 'b'),
+  (2, 'c'),
+  (3, 'a'),
+  (3, 'b'),
+  (3, 'c')]

Issue Links

• links to: [Github] Pull Request #39619 (zhengruifeng)