Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-42497 Support of pandas API on Spark for Spark Connect
  3. SPARK-43078

Separate test into `pyspark-conenct-pandas` and `pyspark-connect-pandas-slow`

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 3.5.0
    • None
    • Connect, Pandas API on Spark
    • None

    Description

      The test `pyspark-connect` takes 2~3 hours due to recently added pandas API on Spark tests, so we'd better to separate pandas API on Spark tests into different test module to reduce the overhead onto `pyspark-connect`.

      Attachments

        Issue Links

          is duplicated by

          Test - A new unit, integration or system test. SPARK-43115 Split pyspark-pandas-connect from pyspark-connect module.

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              itholic Haejoon Lee
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: