Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-37935 Migrate onto error classes
  3. SPARK-47261

Assign classes to Parquet type errors

    XMLWordPrintableJSON

Details

    Description

      Choose a proper name for the error class _LEGACY_ERROR_TEMP_11[72-74] defined in core/src/main/resources/error/error-classes.json. The name should be short but complete (look at the example in error-classes.json).

      Add a test which triggers the error from user code if such test still doesn't exist. Check exception fields by using checkError(). The last function checks valuable error fields only, and avoids dependencies from error text message. In this way, tech editors can modify error format in error-classes.json, and don't worry of Spark's internal tests. Migrate other tests that might trigger the error onto checkError().

      If you cannot reproduce the error from user space (using SQL query), replace the error by an internal error, seeĀ SparkException.internalError().

      Improve the error message format in error-classes.json if the current is not clear. Propose a solution to users how to avoid and fix such kind of errors.

      Please, look at the PR below as examples:

      Attachments

        Issue Links

          Activity

            People

              Junyu Chen junyuc25
              maxgekk Max Gekk
              Max Gekk Max Gekk
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: