[SPARK-29067] divide function does not throw an error, if a number is not passed to it

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.4.3
• Component/s: Java API
• Labels: bulk-closed

Environment

Apache Spark Java : 2.4.3

• Assignee: Unassigned
• Reporter: Mangesh Rananavare
• Created: 12/Sep/19 08:59
• Updated: 25/May/21 01:52
• Resolved: 25/May/21 01:44

Description

Ex. 

dataset.where(col("col1").divide("col2")).$greater(10)).show();

If you see closely I forgot to wrap the divide parameter "col2" into col() function, so basically I pass a String. This should give me a NumberFormatException!, but instead the where clause resolves to null and I get a empty dataset as a result of above computation!!