Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-40005

Self-contained examples with parameter descriptions in PySpark documentation

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Critical
    • Resolution: Done
    • 3.4.0
    • None
    • Documentation, PySpark
    • None

    Description

      This JIRA aims to improve PySpark documentation in:

      • pyspark
      • pyspark.sql
      • pyspark.sql.streaming

      We should:

      If the size of file is large, e.g., dataframe.py, we should split that down into each subtask, and improve documentation.

      Attachments

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-33247 Improve examples and scenarios in docstrings

          • Major - Major loss of function.
          • Resolved
          1.
          		 Make pyspark.sql.group examples self-contained Sub-task Resolved Hyukjin Kwon
          2.
          		 Make pyspark.sql.window examples self-contained Sub-task Resolved Qian Sun
          3.
          		 Make pyspark.sql.dataframe examples self-contained Sub-task Resolved Hyukjin Kwon
          4.
          		 Make pyspark.sql.streaming.readwriter examples self-contained Sub-task Resolved Hyukjin Kwon
          5.
          		 Make pyspark.sql.types examples self-contained Sub-task Resolved Yikun Jiang
          6.
          		 Add Document Parameters for pyspark.sql.window Sub-task Resolved Qian Sun
          7.
          		 Make pyspark.sql.streaming.query examples self-contained Sub-task Resolved Qian Sun
          8.
          		 Make pyspark.sql.catalog examples self-contained Sub-task Resolved Hyukjin Kwon
          9.
          		 Make pyspark.context examples self-contained Sub-task Resolved Ruifeng Zheng
          10.
          		 Make pyspark.sql.column examples self-contained Sub-task Resolved Qian Sun
          11.
          		 Add Document Parameters for pyspark.sql.streaming.query Sub-task Resolved Qian Sun
          12.
          		 Make pyspark.rdd examples self-contained Sub-task Resolved Ruifeng Zheng
          13.
          		 Make pyspark.sql.readwriter examples self-contained Sub-task Resolved Hyukjin Kwon
          14.
          		 Make pyspark.sql.functions examples self-contained Sub-task Resolved Hyukjin Kwon
          15.
          		 Make pyspark.sql.session examples self-contained Sub-task Resolved Hyukjin Kwon
          16.
          		 Make pyspark.sql.window examples self-contained Sub-task Resolved Unassigned
          17.
          		 Make pyspark.files examples self-contained Sub-task Resolved Ruifeng Zheng
          18.
          		 Make pyspark.broadcast examples self-contained Sub-task Resolved Qian Sun
          19.
          		 Make pyspark.taskcontext examples self-contained Sub-task Resolved Hyukjin Kwon
          20.
          		 Make pyspark.resource examples self-contained Sub-task Resolved Hyukjin Kwon
          21.
          		 Remove since usage in streaming/query.py and window.py Sub-task Resolved Hyukjin Kwon

          Activity

            People

              gurwls223 Hyukjin Kwon
              gurwls223 Hyukjin Kwon
              Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: