Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35464

pandas API on Spark: Enable mypy check "disallow_untyped_defs" for main codes.

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.2.0
    • PySpark
    • None

    Description

      Currently many functions in the main codes are still missing type annotations and disabled mypy check "disallow_untyped_defs".

      We should add more type annotations and enable the mypy check.

      Attachments

        Issue Links

          is part of

          Umbrella - An overarching type made of sub-tasks SPARK-34849 SPIP: Support pandas API layer on PySpark

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          relates to

          Sub-task - The sub-task of the issue SPARK-34941 Enable mypy for pandas-on-Spark

          • Major - Major loss of function.
          • Resolved
          1.
          		 Enable disallow_untyped_defs mypy check except for major modules. Sub-task Resolved Takuya Ueshin
          2.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops. Sub-task Resolved Takuya Ueshin
          3.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.spark.accessors. Sub-task Resolved Takuya Ueshin
          4.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.typedef.typehints. Sub-task Resolved Takuya Ueshin
          5.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.accessors. Sub-task Resolved Takuya Ueshin
          6.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.base. Sub-task Resolved Takuya Ueshin
          7.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.frame. Sub-task Resolved Takuya Ueshin
          8.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.generic. Sub-task Resolved Takuya Ueshin
          9.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.window. Sub-task Resolved Kevin Su
          10.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.groupby. Sub-task Resolved Takuya Ueshin
          11.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing. Sub-task Resolved Kevin Su
          12.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.namespace. Sub-task Resolved Takuya Ueshin
          13.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.series. Sub-task Resolved Takuya Ueshin
          14.
          		 Enable disallow_untyped_defs mypy check for pyspark.pandas.utils. Sub-task Resolved Takuya Ueshin
          15.
          		 Refine type hints in pyspark.pandas.window Sub-task Resolved Takuya Ueshin
          16.
          		 Cleanup type hints. Sub-task Resolved Takuya Ueshin

          Activity

            People

              ueshin Takuya Ueshin
              ueshin Takuya Ueshin
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: