[SPARK-31895] df.explain("mode") should work in PySpark side as well for consistency

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: PySpark
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 03/Jun/20 01:01
• Updated: 12/Dec/22 18:10
• Resolved: 03/Jun/20 03:07

Description

Scala:

scala> spark.range(10).explain("cost")
== Optimized Logical Plan ==
Range (0, 10, step=1, splits=Some(12)), Statistics(sizeInBytes=80.0 B)

== Physical Plan ==
*(1) Range (0, 10, step=1, splits=12)

PySpark:

>>> spark.range(10).explain("cost")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.../spark/python/pyspark/sql/dataframe.py", line 333, in explain
    raise TypeError(err_msg)
TypeError: extended (optional) should be provided as bool, got <class 'str'>

Issue Links

• relates to: SPARK-30231 Support explain mode in PySpark df.explain
• links to: [Github] Pull Request #28711 (HyukjinKwon)
• links to: [Github] Pull Request #28711 (HyukjinKwon)