Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-42986

Migrate more PySpark errors onto error classes

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Major
    • Resolution: Resolved
    • 3.5.0
    • None
    • Connect, PySpark
    • None

    Description

      We've done basic error class support/migration for major files on https://issues.apache.org/jira/browse/SPARK-41597.

      Now we should migrate the remaining PySpark errors into error classes.

      Attachments

        1.
        		 Introduce PySparkRuntimeError Sub-task Resolved Haejoon Lee
        2.
        		 Migrate Spark Connect DataFrame errors into error class Sub-task Resolved Haejoon Lee
        3.
        		 Migrate Column errors into error class Sub-task Resolved Haejoon Lee
        4.
        		 Migrate ValueError from DataFrame into PySparkValueError. Sub-task Resolved Haejoon Lee
        5.
        		 Refactoring similar error classes such as `NOT_XXX`. Sub-task Resolved Unassigned
        6.
        		 Automate error class documentation Sub-task Resolved Unassigned
        7.
        		 Testing JVM-captured exceptions from Python side. Sub-task Resolved Unassigned
        8.
        		 Migrate Spark Connect Column errors into error class Sub-task Resolved Haejoon Lee
        9.
        		 Migrate TypeError from DataFrame(Reader|Writer) into error class Sub-task Resolved Haejoon Lee
        10.
        		 Migrate UDF errors into error class Sub-task Resolved Haejoon Lee
        11.
        		 Migrate Expression errors into error class Sub-task Resolved Haejoon Lee
        12.
        		 Introduce PySparkAssersionError Sub-task Resolved Haejoon Lee
        13.
        		 Migrate Structured Streaming errors into error class Sub-task Resolved Haejoon Lee
        14.
        		 Migrate ValueError from Connect DataFrame into error class Sub-task Resolved Haejoon Lee
        15.
        		 Migrate the Spark SQL pandas arrow type errors into error class. Sub-task Resolved Haejoon Lee
        16.
        		 Migrate `TypeError` from Spark SQL types into error class Sub-task Resolved Haejoon Lee
        17.
        		 Migrate Spark Connect Structured Streaming errors into error class Sub-task Resolved Haejoon Lee
        18.
        		 Introduce `PySparkNotImplementError` Sub-task Resolved Haejoon Lee
        19.
        		 Migrate Spark Connect GroupedData error into error class Sub-task Resolved Haejoon Lee
        20.
        		 Migrate Spark Connect Window errors into error class Sub-task Resolved Haejoon Lee
        21.
        		 Migrate Spark Connect session errors into error class Sub-task Resolved Haejoon Lee
        22.
        		 Migrate NotImplementedError into PySparkNotImplementedError Sub-task Resolved Unassigned
        23.
        		 Migrate `ValueError` from Spark SQL types into error class Sub-task Resolved Haejoon Lee
        24.
        		 Migrate PandasUDF value errors into error class Sub-task Resolved Haejoon Lee
        25.
        		 Migrate Spark Connect client errors into error class Sub-task Resolved Haejoon Lee
        26.
        		 Migrate remaining errors from DataFrame(Reader|Writer) into error class Sub-task Resolved Haejoon Lee
        27.
        		 Migrate remaining Connect/SparkSQL errors into error classes Sub-task Resolved Unassigned
        28.
        		 Migrate remaining session errors into error class Sub-task Resolved Haejoon Lee

        Activity

          People

            Unassigned Unassigned
            itholic Haejoon Lee
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: