Derby
  1. Derby
  2. DERBY-2667

Improve diagnostic information for failed derby JUnit tests

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 10.3.1.4
    • Fix Version/s: 10.5.1.1
    • Component/s: Test
    • Labels:
      None

      Description

      Provide a more full featured TestRunner for Derby testing.

      junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.

      1. derby-2667-WriteExceptionsToFileAsTheyHappen.diff
        2 kB
        Kristian Waagan
      2. DERBY-2667_diff_02_21.txt
        2 kB
        Manjula Kutty
      3. DERBY-2667_stat_02_21.txt
        0.1 kB
        Manjula Kutty
      4. DERBY-2667_diff_2_19.txt
        1 kB
        Manjula Kutty
      5. DERBY-2667_stat_02_19.txt
        0.1 kB
        Manjula Kutty
      6. DERBY-2667_diff_02_15.txt
        1 kB
        Manjula Kutty
      7. DERBY-2667_stat_02_15.txt
        0.1 kB
        Manjula Kutty
      8. DERBY-2667_diff_02_06.txt
        1 kB
        Manjula Kutty
      9. DERBY-2667_stat_02_06.txt
        0.1 kB
        Manjula Kutty
      10. JUnitMethodTrace_Extra.diff.txt
        0.8 kB
        Ole Solberg
      11. JUnitMethodTrace.diff.txt
        5 kB
        Ole Solberg
      12. MemRunner.java
        1 kB
        Knut Anders Hatlen
      13. TimeRunner.java
        0.9 kB
        Knut Anders Hatlen

        Issue Links

          Activity

          Hide
          Daniel John Debrunner added a comment -

          Not sure if "robust" is the right tem here, do you really mean a "more informative" test runner?

          junit.textui.TestRunner does in fact print chained exceptions, it's Derby that is not chaining the exceptions. I had a fix that did chain the exceptions but it broke on jdk16 so it was backed out and put into a Jira entry which I think has yet to be fixed. DERBY-2472

          For the separate file issue, have you looked at the way ant runs junit tests? We have the top-level junit-all target and that provides a whole lot of information in XML that can help in diagnosing issues. It's also standard so that others tools know how to handle the information, e.g. CruiseControl presents the ant junit task output as web-pages complete with sections for failed tests etc.

          Show
          Daniel John Debrunner added a comment - Not sure if "robust" is the right tem here, do you really mean a "more informative" test runner? junit.textui.TestRunner does in fact print chained exceptions, it's Derby that is not chaining the exceptions. I had a fix that did chain the exceptions but it broke on jdk16 so it was backed out and put into a Jira entry which I think has yet to be fixed. DERBY-2472 For the separate file issue, have you looked at the way ant runs junit tests? We have the top-level junit-all target and that provides a whole lot of information in XML that can help in diagnosing issues. It's also standard so that others tools know how to handle the information, e.g. CruiseControl presents the ant junit task output as web-pages complete with sections for failed tests etc.
          Hide
          Kathey Marsden added a comment -

          I am having trouble running junit-all and junitreport. (Permissions errors of course). Any idea how to resolve this issue?

          http://www.nabble.com/problem-with-junitreport-target-tf3753226.html#a10606491

          Show
          Kathey Marsden added a comment - I am having trouble running junit-all and junitreport. (Permissions errors of course). Any idea how to resolve this issue? http://www.nabble.com/problem-with-junitreport-target-tf3753226.html#a10606491
          Hide
          Daniel John Debrunner added a comment -

          junitreport has an existing issue DERBY-2234

          I also see some permission problems running junit-all (in cruise control), I was planning on looking into it today to see if it was my environment or not.

          Show
          Daniel John Debrunner added a comment - junitreport has an existing issue DERBY-2234 I also see some permission problems running junit-all (in cruise control), I was planning on looking into it today to see if it was my environment or not.
          Hide
          Mike Matrigali added a comment -

          What I am looking for is a runner that produces text based output that is appropriate for offline nightly test suites. I like to run whatever test runner the systems that are running regression tests against the code line are. Currently I believe this is sun jvm nightly, sun jvm tinderbox and ibm jvm nightly. This was if I match my environment it is likely the tool will work. While it is great
          that people can pick/choose their runner - it would be nice to have at least one runner recommended as the default runner that
          developers could use and nightly regression tests would also use.

          When I started I first tried the ant option, but immediately ran into memory issues, but noticed nightly runs using the other tools
          just worked.

          What I am looking for is:
          o a tool that will run all the tests without failing, and is tested to continue to do so every night.
          o complete set of stack traces, including any nested exceptions. Dan points out this is likely a derby issue
          o the ability to run all the tests and then come back and have a shot at figuring out what went wrong and when. Things
          that would help are:
          o printing each test as it runs. Sometimes when you get errors in teardown the stack gives you no idea what test the teardown
          actually is failing for.
          o preserve the persistent information when a failure occurs. Most important is the derby.log as it may have more information.
          But a copy of the database is also interesting, especially in cases of non reproducible errors where this may have been
          our one chance to gather info, or to allow someone to debug a system specific problem when they don't have access to that
          type of system.
          This may be a test issue rather than the runner. I wonder if we just need to write some generic "tearDown" that on error or
          failure copies current state somewhere

          Show
          Mike Matrigali added a comment - What I am looking for is a runner that produces text based output that is appropriate for offline nightly test suites. I like to run whatever test runner the systems that are running regression tests against the code line are. Currently I believe this is sun jvm nightly, sun jvm tinderbox and ibm jvm nightly. This was if I match my environment it is likely the tool will work. While it is great that people can pick/choose their runner - it would be nice to have at least one runner recommended as the default runner that developers could use and nightly regression tests would also use. When I started I first tried the ant option, but immediately ran into memory issues, but noticed nightly runs using the other tools just worked. What I am looking for is: o a tool that will run all the tests without failing, and is tested to continue to do so every night. o complete set of stack traces, including any nested exceptions. Dan points out this is likely a derby issue o the ability to run all the tests and then come back and have a shot at figuring out what went wrong and when. Things that would help are: o printing each test as it runs. Sometimes when you get errors in teardown the stack gives you no idea what test the teardown actually is failing for. o preserve the persistent information when a failure occurs. Most important is the derby.log as it may have more information. But a copy of the database is also interesting, especially in cases of non reproducible errors where this may have been our one chance to gather info, or to allow someone to debug a system specific problem when they don't have access to that type of system. This may be a test issue rather than the runner. I wonder if we just need to write some generic "tearDown" that on error or failure copies current state somewhere
          Hide
          Daniel John Debrunner added a comment -

          I think most of those requirements are fulfilled by the ant junit task, except for the last one, which as you say is a test issue not a test runner issue.

          I don't see any memory issues running junit-all, I run it as a continuous integration test against Derby. I am getting a few tests failing but have only just started looking into that.

          Note that with the first one I'm looking at the lack of information in trying to solve the failure is not due to a test runner issue, but Derby not providing any information, e.g. I just entered DERBY-2671 for the bug where early failures in starting the network server are lost.

          I think that dependance on a single junit test runner is not a good approach, Derby and the tests themselves should be providing information that can be useful in any test runner. Diversity in test runners may also expose bugs in Derby, e.g. I would say that I only entered DERBY-2671 because I was running in a different test runner to my usual use of the swing one.

          Show
          Daniel John Debrunner added a comment - I think most of those requirements are fulfilled by the ant junit task, except for the last one, which as you say is a test issue not a test runner issue. I don't see any memory issues running junit-all, I run it as a continuous integration test against Derby. I am getting a few tests failing but have only just started looking into that. Note that with the first one I'm looking at the lack of information in trying to solve the failure is not due to a test runner issue, but Derby not providing any information, e.g. I just entered DERBY-2671 for the bug where early failures in starting the network server are lost. I think that dependance on a single junit test runner is not a good approach, Derby and the tests themselves should be providing information that can be useful in any test runner. Diversity in test runners may also expose bugs in Derby, e.g. I would say that I only entered DERBY-2671 because I was running in a different test runner to my usual use of the swing one.
          Hide
          Knut Anders Hatlen added a comment -

          I'm attaching the two test runners I have written and granting licence to ASF in case someone wants to play with them. MemRunner runs all the tests in suites.All and reports the increase/decrease in memory footprint for each fixture. TimeRunner runs all the tests in suites.All and reports how many milliseconds each fixture took.

          Show
          Knut Anders Hatlen added a comment - I'm attaching the two test runners I have written and granting licence to ASF in case someone wants to play with them. MemRunner runs all the tests in suites.All and reports the increase/decrease in memory footprint for each fixture. TimeRunner runs all the tests in suites.All and reports how many milliseconds each fixture took.
          Hide
          Ole Solberg added a comment -

          In the regression testing and also in more experimental testing I have felt a need for more details in the JUnit test reports - e.g. to convince myself that a test was really executed.
          I have done this by modifying 'java/testing/org/apache/derbyTesting/junit/BaseTestCase.java' to print the test method name, and have also included a printout of the time (in ms.) used by the test method.
          I use "-Dderby.tests.trace=true" to turn this on.
          Attachement JUnitMethodTrace.diff.txt.

          'java/testing/org/apache/derbyTesting/unitTests/junit/FormatableBitSetTest.java' currently uses JUnit TestCase and would thus not give the desired extra information: The JUnitMethodTrace_Extra.diff.txt patch changes that by using 'BaseTestCase'.
          Attachement JUnitMethodTrace_Extra.diff.txt

          Example output:
          .
          testSetCharacterStream used 200ms .
          testGetAsciiStream used 664ms .
          testGetCharacterStream used 638ms .
          testGetCharacterStreamWithUnicode used 817ms .
          testTriggersWithClobColumn used 2453ms .
          testGetSubString used 431ms E.
          testGetSubStringWithUnicode used 855ms .
          testPositionString used 713ms .
          testPositionStringWithUnicode used 739ms .
          testPositionClob used 4204ms .
          testPositionClobWithUnicode used 4832ms .
          testSmallClobFields used 82ms .
          testGetClobFromIntColumn used 302ms F.
          testSetClobToIntColumn used 412ms F.
          testRaisingOfExceptionsClob used 523ms F.
          testSetClob used 1116ms .
          testPositionAgressive used 9073ms .
          testClobAfterClose used 587ms .
          testLockingClob used 60695ms .
          testLockingWithLongRowClob used 60178ms .
          .
          .
          .

          Show
          Ole Solberg added a comment - In the regression testing and also in more experimental testing I have felt a need for more details in the JUnit test reports - e.g. to convince myself that a test was really executed. I have done this by modifying 'java/testing/org/apache/derbyTesting/junit/BaseTestCase.java' to print the test method name, and have also included a printout of the time (in ms.) used by the test method. I use "-Dderby.tests.trace=true" to turn this on. Attachement JUnitMethodTrace.diff.txt. 'java/testing/org/apache/derbyTesting/unitTests/junit/FormatableBitSetTest.java' currently uses JUnit TestCase and would thus not give the desired extra information: The JUnitMethodTrace_Extra.diff.txt patch changes that by using 'BaseTestCase'. Attachement JUnitMethodTrace_Extra.diff.txt Example output: . testSetCharacterStream used 200ms . testGetAsciiStream used 664ms . testGetCharacterStream used 638ms . testGetCharacterStreamWithUnicode used 817ms . testTriggersWithClobColumn used 2453ms . testGetSubString used 431ms E. testGetSubStringWithUnicode used 855ms . testPositionString used 713ms . testPositionStringWithUnicode used 739ms . testPositionClob used 4204ms . testPositionClobWithUnicode used 4832ms . testSmallClobFields used 82ms . testGetClobFromIntColumn used 302ms F. testSetClobToIntColumn used 412ms F. testRaisingOfExceptionsClob used 523ms F. testSetClob used 1116ms . testPositionAgressive used 9073ms . testClobAfterClose used 587ms . testLockingClob used 60695ms . testLockingWithLongRowClob used 60178ms . . . .
          Hide
          Knut Anders Hatlen added a comment -

          Thanks Ole! This looks useful to me.

          Committed JUnitMethodTrace.diff.txt with revision 552046 after resolving a conflict with another commit in TestConfiguration. I also removed the two new instance variables in BaseTestCase since testCaseName could be accessed through getName(), and the call to TestConfiguration.getCurrent().doTrace() should happen in runBare() (calling it from the constructor could cause the wrong TestConfiguration object to be used).

          Show
          Knut Anders Hatlen added a comment - Thanks Ole! This looks useful to me. Committed JUnitMethodTrace.diff.txt with revision 552046 after resolving a conflict with another commit in TestConfiguration. I also removed the two new instance variables in BaseTestCase since testCaseName could be accessed through getName(), and the call to TestConfiguration.getCurrent().doTrace() should happen in runBare() (calling it from the constructor could cause the wrong TestConfiguration object to be used).
          Hide
          Knut Anders Hatlen added a comment -

          Committed JUnitMethodTrace_Extra.diff.txt with revision 552047.

          Show
          Knut Anders Hatlen added a comment - Committed JUnitMethodTrace_Extra.diff.txt with revision 552047.
          Hide
          Manjula Kutty added a comment -

          Attaching a patch which will copy the derby.log file of the failed test to the dir user.dir/fail/clientname/testclass/testname. Please review and if looks useful please commit

          Show
          Manjula Kutty added a comment - Attaching a patch which will copy the derby.log file of the failed test to the dir user.dir/fail/clientname/testclass/testname. Please review and if looks useful please commit
          Hide
          Daniel John Debrunner added a comment -

          It would be good to add comments to the code so that it's clear to readers what the purpose of the code is.

          The streams for the files don't seem to get closed.

          The caught exception is never re-thrown, so this will hide any failures.

          Fetching the property user.dir is likely to result in a security exception since it's not in a priv block,
          the class has a utility method for reading system properties, but it's not needed since one does
          not need to read user.dir to access files in the current directory. Something like new File("system", "derby.log") should work.

          Also it would be good to test that the file derby.log exists, it's not guaranteed that a test will create it.

          Also probably better not to construct file paths using File.separator, instead use File objects.

          Show
          Daniel John Debrunner added a comment - It would be good to add comments to the code so that it's clear to readers what the purpose of the code is. The streams for the files don't seem to get closed. The caught exception is never re-thrown, so this will hide any failures. Fetching the property user.dir is likely to result in a security exception since it's not in a priv block, the class has a utility method for reading system properties, but it's not needed since one does not need to read user.dir to access files in the current directory. Something like new File("system", "derby.log") should work. Also it would be good to test that the file derby.log exists, it's not guaranteed that a test will create it. Also probably better not to construct file paths using File.separator, instead use File objects.
          Hide
          Dyre Tjeldvoll added a comment -

          Removing the patch flag since the latest review comments indicate that another rev of the patch is required.

          Show
          Dyre Tjeldvoll added a comment - Removing the patch flag since the latest review comments indicate that another rev of the patch is required.
          Hide
          Manjula Kutty added a comment -

          Thanks for the comments Dan. Please review the new patch attached. I hope I answered all your comments in that patch. Please review and if everything looks good, please commit

          Show
          Manjula Kutty added a comment - Thanks for the comments Dan. Please review the new patch attached. I hope I answered all your comments in that patch. Please review and if everything looks good, please commit
          Hide
          Knut Anders Hatlen added a comment -

          The check of testDir.exists() looks misplaced. If it returns false, the construction of a FileInputStream would have failed with FileNotFoundException two lines above it.

          Also, I think it would be best only to create the FileOutputStream if testDir exists. Otherwise, we'll create an empty file for each failing test that doesn't have a derby.log file.

          Many of the lines use a mix of tabs and spaces for indentation.

          Should new File("System", "derby.log"); use "system" instead? I don't think it will work on case-sensitive file systems, since the directory created by the framework is called "system".

          Show
          Knut Anders Hatlen added a comment - The check of testDir.exists() looks misplaced. If it returns false, the construction of a FileInputStream would have failed with FileNotFoundException two lines above it. Also, I think it would be best only to create the FileOutputStream if testDir exists. Otherwise, we'll create an empty file for each failing test that doesn't have a derby.log file. Many of the lines use a mix of tabs and spaces for indentation. Should new File("System", "derby.log"); use "system" instead? I don't think it will work on case-sensitive file systems, since the directory created by the framework is called "system".
          Hide
          Myrna van Lunteren added a comment -

          I am wondering why you choose to only catch Exceptions?
          For instance, a 'normal' failure would be likely a Throwable, e.g. a junit.framework.AssertionFailedError...
          Maybe it makes sense to save derby.log when a Throwable is encountered?

          Show
          Myrna van Lunteren added a comment - I am wondering why you choose to only catch Exceptions? For instance, a 'normal' failure would be likely a Throwable, e.g. a junit.framework.AssertionFailedError... Maybe it makes sense to save derby.log when a Throwable is encountered?
          Hide
          Manjula Kutty added a comment -

          Thanks for the comments. Please find the modified patch and I hope I tried to address all your comments in that patch. Please review

          Show
          Manjula Kutty added a comment - Thanks for the comments. Please find the modified patch and I hope I tried to address all your comments in that patch. Please review
          Hide
          Knut Anders Hatlen added a comment -

          Is the check for (running != null) necessary? I don't see how it could be null.

          First two lines use spaces for indentation, the rest use tabs. Would be good to use one of them consistently. Since the surrounding code uses spaces, I think that's the best option.

          Should the call to getFailureFolder() also be moved inside the if? Then we don't create a folder unless we have to.

          Would it make sense to put a try/finally block inside the catch block and re-throw the exception in the finally clause? This way we make sure that the original exception is not hidden if saving derby.log fails for some reason.

          Show
          Knut Anders Hatlen added a comment - Is the check for (running != null) necessary? I don't see how it could be null. First two lines use spaces for indentation, the rest use tabs. Would be good to use one of them consistently. Since the surrounding code uses spaces, I think that's the best option. Should the call to getFailureFolder() also be moved inside the if? Then we don't create a folder unless we have to. Would it make sense to put a try/finally block inside the catch block and re-throw the exception in the finally clause? This way we make sure that the original exception is not hidden if saving derby.log fails for some reason.
          Hide
          Manjula Kutty added a comment -

          Thanks for your comments. Here is the new patch. Please ereview

          Show
          Manjula Kutty added a comment - Thanks for your comments. Here is the new patch. Please ereview
          Hide
          Myrna van Lunteren added a comment - - edited

          Thanks Manjula, I like the latest patch.

          I committed patch DERBY-2667_diff_02_21.txt with revision 630077.

          Probably superfluous, but, I do want to point out that the TestConfiguration.getFailureFolder creates a 'fail' folder, and all directories underneath (fail/<configuration>/<testName>/<failedFixtureName>).
          This means, that if you run a subsequent failing fixture test in the same top level directory, previous derby.logs will be overwritten, and also, old run's fail directories won't get cleaned out.

          Show
          Myrna van Lunteren added a comment - - edited Thanks Manjula, I like the latest patch. I committed patch DERBY-2667 _diff_02_21.txt with revision 630077. Probably superfluous, but, I do want to point out that the TestConfiguration.getFailureFolder creates a 'fail' folder, and all directories underneath (fail/<configuration>/<testName>/<failedFixtureName>). This means, that if you run a subsequent failing fixture test in the same top level directory, previous derby.logs will be overwritten, and also, old run's fail directories won't get cleaned out.
          Hide
          Knut Anders Hatlen added a comment -

          Thanks for addressing my comments. Please double check the indentation settings in your IDE, though. The latest patch reintroduced the strange indentation from the earlier patches (each line starts with eight space characters, but for some reason tabs are appended if more indentation is needed).

          Show
          Knut Anders Hatlen added a comment - Thanks for addressing my comments. Please double check the indentation settings in your IDE, though. The latest patch reintroduced the strange indentation from the earlier patches (each line starts with eight space characters, but for some reason tabs are appended if more indentation is needed).
          Hide
          Kathey Marsden added a comment -

          In eclipse I see a warning for this finally block
          finally

          { throw running; }

          }

          finally block does not complete normally.

          I think this is because we lose any exception that occurs in saving off the derby.log file.
          I assume though that that was intentional because it is more important to preserve the
          exception that occurred running the test, than the one that occurred saving the file. Is
          that correct. I found this thread about the warning:
          http://dev.eclipse.org/newslists/news.eclipse.tools.jdt/msg10787.html

          Show
          Kathey Marsden added a comment - In eclipse I see a warning for this finally block finally { throw running; } } finally block does not complete normally. I think this is because we lose any exception that occurs in saving off the derby.log file. I assume though that that was intentional because it is more important to preserve the exception that occurred running the test, than the one that occurred saving the file. Is that correct. I found this thread about the warning: http://dev.eclipse.org/newslists/news.eclipse.tools.jdt/msg10787.html
          Hide
          Manjula Kutty added a comment -

          Yes, that was my intention. But no harm if I can catch the exception from saving the file too. Do you have any good suggestion for that?

          Show
          Manjula Kutty added a comment - Yes, that was my intention. But no harm if I can catch the exception from saving the file too. Do you have any good suggestion for that?
          Hide
          Kathey Marsden added a comment -

          No suggestion. I just hadn't seen the warning before so wanted to make sure I understood what it meant.

          Show
          Kathey Marsden added a comment - No suggestion. I just hadn't seen the warning before so wanted to make sure I understood what it meant.
          Hide
          Kristian Waagan added a comment -

          'derby-2667-WriteExceptionsToFileAsTheyHappen.diff' is a patch that makes BaseTestCase also write the stack trace to file when an error occurs. The file will be written to the same location as derby.log, and currently it will overwrite any existing file.
          I have only tested it lightly, but it seems to work fine for me. Is it something we could consider adding?

          The reason why I wrote it, is that it is sometimes very annoying to wait for suites.All to finish when you get an error/failure on the first line of dots...

          Show
          Kristian Waagan added a comment - 'derby-2667-WriteExceptionsToFileAsTheyHappen.diff' is a patch that makes BaseTestCase also write the stack trace to file when an error occurs. The file will be written to the same location as derby.log, and currently it will overwrite any existing file. I have only tested it lightly, but it seems to work fine for me. Is it something we could consider adding? The reason why I wrote it, is that it is sometimes very annoying to wait for suites.All to finish when you get an error/failure on the first line of dots...
          Hide
          Dag H. Wanvik added a comment -

          Re derby-2667-WriteExceptionsToFileAsTheyHappen.diff: Why not append to the fle instead of overwriting? I tend to find the first error/failure the most interesting...

          Show
          Dag H. Wanvik added a comment - Re derby-2667-WriteExceptionsToFileAsTheyHappen.diff: Why not append to the fle instead of overwriting? I tend to find the first error/failure the most interesting...
          Hide
          Kristian Waagan added a comment -

          Well, maybe I misunderstood, but if you overwrite, it will be because you run the test / suite again, won't it?
          The naming logic of the failure folder seems to consist of something related to connection / JDBCClient, the test class and the test method.
          So as an example:
          ./Embedded_40/BlobClob4BlobTest/testBlobAfterClosingConnection/stacktrace.out

          I just assumed there could be only one exception thrown for a single run (in runBare), but maybe my comment above regarding overwriting was a bit confusing.
          I haven't actually studied the logic for the failure folder naming, but maybe someone else has some info?

          But of course, appending to the file might still be a good thing to do, or maybe a timestamp or something in the filename?
          (if you tend to run the tests from the same location over and over again when testing/debugging)

          Show
          Kristian Waagan added a comment - Well, maybe I misunderstood, but if you overwrite, it will be because you run the test / suite again, won't it? The naming logic of the failure folder seems to consist of something related to connection / JDBCClient, the test class and the test method. So as an example: ./Embedded_40/BlobClob4BlobTest/testBlobAfterClosingConnection/stacktrace.out I just assumed there could be only one exception thrown for a single run (in runBare), but maybe my comment above regarding overwriting was a bit confusing. I haven't actually studied the logic for the failure folder naming, but maybe someone else has some info? But of course, appending to the file might still be a good thing to do, or maybe a timestamp or something in the filename? (if you tend to run the tests from the same location over and over again when testing/debugging)
          Hide
          Dag H. Wanvik added a comment -

          Thanks, Kristian, I was not thinking of the multiple runs case. As long as the first failure/error
          in one run is not clobbered by a subsequent one in the same run, I am OK with this solution.

          Show
          Dag H. Wanvik added a comment - Thanks, Kristian, I was not thinking of the multiple runs case. As long as the first failure/error in one run is not clobbered by a subsequent one in the same run, I am OK with this solution.
          Hide
          Kristian Waagan added a comment -

          Dag, it is a valid concern and I have not actually verified that no files are overwritten in all configurations.
          People should feel free to do this, or we could just do the easy thing and add a timestamp and append to the file.

          Show
          Kristian Waagan added a comment - Dag, it is a valid concern and I have not actually verified that no files are overwritten in all configurations. People should feel free to do this, or we could just do the easy thing and add a timestamp and append to the file.
          Hide
          Kathey Marsden added a comment -

          Several improvements have been made under this issue over several releases, actually none to the TestRunner itself. I am thinking resolving it now so more specific issues can be opened and resolved in the release where they were fixed. There is one outstanding patch from Kristian that would be very useful, but the patch no longer applies to trunk.

          Should I leave this open until that makes it in, or just close this one out and open a new one to print the failure trace to the fail directory as they happen?

          Show
          Kathey Marsden added a comment - Several improvements have been made under this issue over several releases, actually none to the TestRunner itself. I am thinking resolving it now so more specific issues can be opened and resolved in the release where they were fixed. There is one outstanding patch from Kristian that would be very useful, but the patch no longer applies to trunk. Should I leave this open until that makes it in, or just close this one out and open a new one to print the failure trace to the fail directory as they happen?
          Hide
          Kristian Waagan added a comment -

          I'd say close this one and create a new one for the second feature.
          Once it is logged, I'll refresh the patch and ready it for review.

          Show
          Kristian Waagan added a comment - I'd say close this one and create a new one for the second feature. Once it is logged, I'll refresh the patch and ready it for review.
          Hide
          Kathey Marsden added a comment -

          Changing title to more accurately reflect what was done under this issue.

          Show
          Kathey Marsden added a comment - Changing title to more accurately reflect what was done under this issue.
          Hide
          Kathey Marsden added a comment -

          Closing this issue. I will open up another one for the pending patch.

          Show
          Kathey Marsden added a comment - Closing this issue. I will open up another one for the pending patch.

            People

            • Assignee:
              Unassigned
              Reporter:
              Kathey Marsden
            • Votes:
              2 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development