[SPARK-42017] df["bad_key"] does not raise AnalysisException

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 4.0.0
• Component/s: Connect, Tests
• Assignee: Unassigned
• Reporter: Hyukjin Kwon
• Created: 12/Jan/23 05:41
• Updated: 24/Aug/23 00:48
• Resolved: 24/Aug/23 00:48

Description

e.g.)

23/01/12 14:33:43 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
FAILED [  8%]
pyspark/sql/tests/test_column.py:105 (ColumnParityTests.test_access_column)
self = <pyspark.sql.tests.connect.test_parity_column.ColumnParityTests testMethod=test_access_column>

    def test_access_column(self):
        df = self.df
        self.assertTrue(isinstance(df.key, Column))
        self.assertTrue(isinstance(df["key"], Column))
        self.assertTrue(isinstance(df[0], Column))
        self.assertRaises(IndexError, lambda: df[2])
>       self.assertRaises(AnalysisException, lambda: df["bad_key"])
E       AssertionError: AnalysisException not raised by <lambda>

../test_column.py:112: AssertionError

Issue Links

• links to: [Github] Pull Request #42608 (zhengruifeng)