Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-42828

PySpark type hint returns Any for methods on GroupedData

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.3.0, 3.3.1, 3.3.2
    • 3.5.0
    • PySpark
    • None

    Description

      Since upgrading to PySpark 3.3.x, type hints for

      df.groupBy(...).count()

      are now returning Any instead of DataFrame, causing type inference issues downstream. This used to be correctly typed prior to 3.3.x.

      Attachments

        Activity

          People

            apachespark Apache Spark
            j03wang Joe Wang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: