Lucene - Core
  1. Lucene - Core
  2. LUCENE-5577

Temporary folder and file management (and cleanup facilities)

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.8, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      This is a spinoff of the work Mark Miller, Uwe and me have initiated in SOLR-5914.

      The core concept is this:

      • every test should create its temporary folders and files in a sub-folder of one, easily identifiable, parent.
      • the parent folder is named after the test class, with [org.apache.] removed for brevity. the folder name includes the master seed and a generated sequence integer, if needed, to easily demarcate different runs (and identify which run produced which files).
      • temporary folder/ file creation routines should be available in LTC, so that no additional imports/ fiddling is needed.
        LTC#createTempDir()
        LTC#createTempDir(String prefix)
        LTC#createTempFile()
        LTC#createTempFile(String prefix, String suffix)
        

        Note the absence of "create a temporary file/dir under a given directory" method. These shouldn't be needed since any temporary folder created by the above is guaranteed to be empty. If one still needs temporary file creation use createTempDir() and then relevant File class methods.

      • any temporary folders and files should be removable at the end of the suite (class), if all tests completes successfully. Failure to remove any files should be marked as an error (will manifest itself on windows if one leaves open file handles).
      • there should be a way to temporarily mark a class to circumvent the above check (SuppressTempFileChecks annotation for known offenders). Annotating a class with SuppressTempFileChecks will still attempt to remove temporary files, but will not cause an error if this fails. Any files that couldn't be deleted will be printed to stdout with an appropriate message.
      • there should be a way for developers to leave temporary files on disk for further inspection (even in case of successful runs). The following system properties will work:
                "tests.leavetmpdir" /* default */,
                "tests.leaveTemporary" /* ANT tasks's (junit4) flag. */,
                "tests.leavetemporary" /* lowercase version of the above */,
                "solr.test.leavetmpdir" /* Solr's legacy property for backcompat */))
        

        Issue Links

          Activity

          Hide
          Dawid Weiss added a comment -

          I think the code is pretty much ready (against trunk). I left it in SVN at:

          https://svn.apache.org/repos/asf/lucene/dev/branches/solr5914

          Would like to commit shortly, it's a fairly sizeable patch and I wouldn't want it to go out of sync.

          Show
          Dawid Weiss added a comment - I think the code is pretty much ready (against trunk). I left it in SVN at: https://svn.apache.org/repos/asf/lucene/dev/branches/solr5914 Would like to commit shortly, it's a fairly sizeable patch and I wouldn't want it to go out of sync.
          Hide
          Mark Miller added a comment -

          Have not looked at the code yet, but +1 beautiful.

          Show
          Mark Miller added a comment - Have not looked at the code yet, but +1 beautiful.
          Hide
          ASF subversion and git services added a comment -

          Commit 1585028 from dweiss@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1585028 ]

          LUCENE-5577: Temporary folder and file management (and cleanup facilities)

          Show
          ASF subversion and git services added a comment - Commit 1585028 from dweiss@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1585028 ] LUCENE-5577 : Temporary folder and file management (and cleanup facilities)
          Hide
          ASF subversion and git services added a comment -

          Commit 1585029 from dweiss@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1585029 ]

          LUCENE-5577: Added javadocs and minor renames.

          Show
          ASF subversion and git services added a comment - Commit 1585029 from dweiss@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1585029 ] LUCENE-5577 : Added javadocs and minor renames.
          Hide
          Dawid Weiss added a comment -

          Mark, Uwe - do you know how to backport this to 4.x other than by manual labor? The patch is pretty extensive and I'm not really familiar with svn (looking for something like git's rebase).

          Show
          Dawid Weiss added a comment - Mark, Uwe - do you know how to backport this to 4.x other than by manual labor? The patch is pretty extensive and I'm not really familiar with svn (looking for something like git's rebase).
          Hide
          Dawid Weiss added a comment -

          Nevermind, got it.

          Show
          Dawid Weiss added a comment - Nevermind, got it.
          Hide
          ASF subversion and git services added a comment -

          Commit 1585035 from dweiss@apache.org in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1585035 ]

          LUCENE-5577: Temporary folder and file management (and cleanup facilities)

          Show
          ASF subversion and git services added a comment - Commit 1585035 from dweiss@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1585035 ] LUCENE-5577 : Temporary folder and file management (and cleanup facilities)
          Hide
          ASF subversion and git services added a comment -

          Commit 1585052 from dweiss@apache.org in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1585052 ]

          LUCENE-5577: the cleanup code didn't cleanup after itself...

          Show
          ASF subversion and git services added a comment - Commit 1585052 from dweiss@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1585052 ] LUCENE-5577 : the cleanup code didn't cleanup after itself...
          Hide
          ASF subversion and git services added a comment -

          Commit 1585053 from dweiss@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1585053 ]

          LUCENE-5577: the cleanup code didn't cleanup after itself...

          Show
          ASF subversion and git services added a comment - Commit 1585053 from dweiss@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1585053 ] LUCENE-5577 : the cleanup code didn't cleanup after itself...
          Hide
          Uwe Schindler added a comment -

          Close issue after release of 4.8.0

          Show
          Uwe Schindler added a comment - Close issue after release of 4.8.0

            People

            • Assignee:
              Dawid Weiss
              Reporter:
              Dawid Weiss
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development