Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1798

Tests should clean up temp files

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 0.9.1
    • 1.0.0
    • Build
    • None

    Description

      Three issues related to temp files that tests generate – these should be touched up for hygiene but are not urgent.

      Modules have a log4j.properties which directs the unit-test.log output file to a directory like [module]/target/unit-test.log. But this ends up creating [module]/[module]/target/unit-test.log instead of former.

      The work/ directory is not deleted by "mvn clean", in the parent and in modules. Neither is the checkpoint/ directory created under the various external modules.

      Many tests create a temp directory, which is not usually deleted. This can be largely resolved by calling deleteOnExit() at creation and trying to call Utils.deleteRecursively consistently to clean up, sometimes in an "@After" method.

      (If anyone seconds the motion, I can create a more significant change that introduces a new test trait along the lines of LocalSparkContext, which provides management of temp directories for subclasses to take advantage of.)

      Attachments

        Activity

          People

            srowen Sean R. Owen
            srowen Sean R. Owen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: