[SPARK-45611] spark.python.pyspark.sql.functions Typo at date_format Function

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 4.0.0
• Component/s: PySpark
• Labels: pull-request-available
• Assignee: Mete Can Akar
• Reporter: Mete Can Akar
• Created: 19/Oct/23 17:42
• Updated: 20/Oct/23 08:37
• Resolved: 20/Oct/23 00:12

Description

In the spark.python.pyspark.sql.functions module, at the date_format method's doctest, there is a typo in the year format.

Instead of 'MM/dd/yyy', it should be 'MM/dd/yyyy' as the expected output[Row(date='04/08/2015')] indicates the following format "MM/dd/yyyy".

 

From the official documentation:

df = spark.createDataFrame([('2015-04-08',)], ['dt'])
df.select(date_format('dt', 'MM/dd/yyy').alias('date')).collect()
[Row(date='04/08/2015')]

 

As a solution, I proposed the PR https://github.com/apache/spark/pull/43442.

Issue Links

• links to: GitHub Pull Request #43442