Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10380

Confusing examples in pyspark SQL docs

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.0.0
    • Documentation, SQL

    Description

      There’s an error in the astype() documentation, as it uses cast instead of astype. It should probably include a mention that astype is an alias for cast (and vice versa in the cast documentation): https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.Column.astype

      The same error occurs with drop_duplicates and dropDuplicates: https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame.drop_duplicates

      The issue here is we are copying the code. According to davies the easiest way is to copy the method and just add new docs.

      Attachments

        Activity

          People

            rxin Reynold Xin
            marmbrus Michael Armbrust
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: