[SPARK-41811] Implement SparkSession.sql's string formatter

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Connect
• Labels: pull-request-available
• Epic Link: Spark Connect
• Assignee: Ruifeng Zheng
• Reporter: Hyukjin Kwon
• Created: 02/Jan/23 07:24
• Updated: 11/Apr/24 20:37
• Resolved: 11/Apr/24 00:06

Description

**********************************************************************
File "/.../spark/python/pyspark/sql/connect/session.py", line 345, in pyspark.sql.connect.session.SparkSession.sql
Failed example:
    spark.sql(
        "SELECT * FROM range(10) WHERE id > {bound1} AND id < {bound2}", bound1=7, bound2=9
    ).show()
Exception raised:
    Traceback (most recent call last):
      File "/.../miniconda3/envs/python3.9/lib/python3.9/doctest.py", line 1336, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest pyspark.sql.connect.session.SparkSession.sql[1]>", line 1, in <module>
        spark.sql(
    TypeError: sql() got an unexpected keyword argument 'bound1'
**********************************************************************
File "/.../spark/python/pyspark/sql/connect/session.py", line 355, in pyspark.sql.connect.session.SparkSession.sql
Failed example:
    spark.sql(
        "SELECT {col} FROM {mydf} WHERE id IN {x}",
        col=mydf.id, mydf=mydf, x=tuple(range(4))).show()
Exception raised:
    Traceback (most recent call last):
      File "/.../miniconda3/envs/python3.9/lib/python3.9/doctest.py", line 1336, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest pyspark.sql.connect.session.SparkSession.sql[3]>", line 1, in <module>
        spark.sql(
    TypeError: sql() got an unexpected keyword argument 'col'

Issue Links

• is blocked by: SPARK-42402 Support parameterized SQL by sql()
• links to: GitHub Pull Request #45277
• links to: GitHub Pull Request #45614