Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20689

python doctest leaking bucketed table

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.0
    • 2.3.0
    • PySpark, SQL
    • None

    Description

      When trying to address build test failure in SPARK-20661 we discovered some tables are unexpectedly left behind causing R tests to fail. While we changed the R tests to be more resilient, we investigated further to see what was creating those tables.

      It turns out pyspark doctest is calling saveAsTable without ever dropping them. Since we have separate python tests for bucketed table, and we don't check for result in doctest, there is really no need to run the doctest

      Attachments

        Activity

          People

            felixcheung Felix Cheung
            felixcheung Felix Cheung
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: