Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-36845

Inline type hint files for files in python/pyspark/sql

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Major
    • Resolution: Done
    • 3.3.0
    • None
    • PySpark, SQL
    • None

    Description

      Currently there are type hint stub files (*.pyi) to show the expected types for functions, but we can also take advantage of static type checking within the functions by inlining the type hints.

      Attachments

        Issue Links

          is related to

          Umbrella - An overarching type made of sub-tasks SPARK-37093 Inline type hints python/pyspark/streaming

          • Major - Major loss of function.
          • Resolved

          Umbrella - An overarching type made of sub-tasks SPARK-37094 Inline type hints for files in python/pyspark

          • Major - Major loss of function.
          • Resolved

          Umbrella - An overarching type made of sub-tasks SPARK-37395 Inline type hint files for files in python/pyspark/ml

          • Major - Major loss of function.
          • Resolved

          Umbrella - An overarching type made of sub-tasks SPARK-37396 Inline type hint files for files in python/pyspark/mllib

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-36145 Remove Python 3.6 support in codebase and CI

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Umbrella - An overarching type made of sub-tasks SPARK-37395 Inline type hint files for files in python/pyspark/ml

          • Major - Major loss of function.
          • Resolved

          Umbrella - An overarching type made of sub-tasks SPARK-37396 Inline type hint files for files in python/pyspark/mllib

          • Major - Major loss of function.
          • Resolved
          1.
          		 Inline most of type hint files under pyspark/sql/pandas folder Sub-task Resolved Takuya Ueshin
          2.
          		 Inline type hints for python/pyspark/sql/functions.py Sub-task Resolved Xinrong Meng
          3.
          		 Inline type hints for python/pyspark/sql/catalog.py Sub-task Resolved Xinrong Meng
          4.
          		 Inline type hints for python/pyspark/sql/session.py Sub-task Resolved Takuya Ueshin
          5.
          		 Inline type hints for python/pyspark/sql/dataframe.py Sub-task Resolved Apache Spark
          6.
          		 Inline type hints for python/pyspark/sql/context.py Sub-task Resolved dch nguyen
          7.
          		 Inline type hints for python/pyspark/sql/conf.py Sub-task Resolved Unassigned
          8.
          		 Replace collections.namedtuple() by typing.NamedTuple Sub-task Resolved Xinrong Meng
          9.
          		 Inline type hints for conf.py and observation.py in python/pyspark/sql Sub-task Resolved Xinrong Meng
          10.
          		 Inline type hints for python/pyspark/sql/types.py Sub-task Resolved Xinrong Meng
          11.
          		 Inline type hints for python/pyspark/sql/window.py Sub-task Resolved Xinrong Meng
          12.
          		 Inline type hints for group.py in python/pyspark/sql Sub-task Resolved dch nguyen
          13.
          		 Inline type hints for python/pyspark/sql/avro/functions.py Sub-task Resolved Xinrong Meng
          14.
          		 Inline type hints for python/pyspark/sql/readwriter.py Sub-task Resolved Xinrong Meng
          15.
          		 Remove unused python/pyspark/sql/__init__.pyi Sub-task Resolved dch nguyen
          16.
          		 Inline type hints for python/pyspark/sql/udf.py Sub-task Resolved dch nguyen
          17.
          		 Inline type hints for python/pyspark/sql/column.py Sub-task Resolved Xinrong Meng
          18.
          		 Inline type hints for python/pyspark/resource/information.py and python/pyspark/resource/profile.py Sub-task Resolved Unassigned
          19.
          		 Inline type hints for spark/python/pyspark/sql/streaming.py Sub-task Resolved Xinrong Meng
          20.
          		 Add type hints to python/pyspark/sql/util.py Sub-task Resolved Takuya Ueshin
          21.
          		 Inline type hints for python/pyspark/resource/requests.py Sub-task Resolved dch nguyen
          22.
          		 Clean up inlining type hints under SQL module Sub-task Resolved Takuya Ueshin

          Activity

            People

              XinrongM Xinrong Meng
              ueshin Takuya Ueshin
              Takuya Ueshin Takuya Ueshin
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: