Derby
  1. Derby
  2. DERBY-5649

make improvements to nstest so it's easier to run/analyze/debug

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 10.9.1.0
    • Fix Version/s: 10.8.3.0, 10.9.1.0
    • Component/s: Test
    • Labels:
      None

      Description

      The system test nstest ran into a number of error situations during the 10.8.2 QA cycle. However, some are known, some were found to be pre-existing situations (although intermittent, so we've been lucky/unlucky not to run into them). Some errors are expected. And some problems were indeed new.
      However, the test output is very wordy and it's complicated identifying real issues and sorting through the messages.
      It would be helpful to clean this up some.

      I found the following areas for easy improvement:

      • InitThread messages and Intializer.java: exited add_one_row: 1 rows
        seems like this is really the same message.
        If we eliminate one, we'll have limited that part of the output by 50%.
      • TesterThreads - seems to send one message re 'attempting to', one for fail/success.
        Again, if we eliminate the 'attempted to' messages, we'd have made the output smaller.
      • db_util.pick_one - seems also unnecessary - can this be combined with the
        insert / update / delete messages that are using the picked row?
      • ERROR 22003 -> The resulting value is ourside the range for data type DECIMAL/NUMERIC(5,0)
        The column is t_decimal(decimal), i.e. a decimal(5,0). The value it's attempting to insert is clearly
        not suitable. From the code (in dbUtil) it does not look like this was intended to be a negative test.
        Eliminating the error (and its corresponding stack prints) would probably make the output considerably smaller
        and make looking for interesting errors easier.
      • There seems to be a section that can be used as a smaller test case, but you need to comment out the 'normal' settings, and uncomment out these settings. It would make more sense to make the small configuration as an option.
      • With a small configuration, the backup thread would run on when all other tests are done, because it has the same
        value for MAX_ITERATIONS, but in contrast to the tester threads, the backup threads runs every 10 minutes.
        Thus, when all other threads are done, the backup threads continue until 50x10 minutes have passed (plus the time it takes to actually do the backup, re-encrypt, restore). So it would make more sense to finish the backup threads if there is no further activity to the database.
      • there are some typos and strange line-spacings making some comments hard to read.
      1. DERBY-5649.diff1
        23 kB
        Myrna van Lunteren

        Issue Links

          Activity

          Hide
          Myrna van Lunteren added a comment -

          I made changes for this with revision 1301290 to trunk and 1301294 to 10.8.

          Show
          Myrna van Lunteren added a comment - I made changes for this with revision 1301290 to trunk and 1301294 to 10.8.
          Hide
          Myrna van Lunteren added a comment -

          I'm reopening this, to adjust the test a little more;
          There's an option to run without a backup thread by setting the property -Dderby.nstest.backupRestore=false, but when you do this, the test runs into an ArrayIndexOutOfBounds error on creating the tester threads, which then confuses the workings of the main method so it won't finish cleanly.
          In that case, the Memcheck keeps going, so just to be on the safe side, I'll add a check for remaining test threads in the Memcheck.

          While I'm add it, I'll remove some commented-out settings that appear to be left behind from previous configuration attempts.

          Show
          Myrna van Lunteren added a comment - I'm reopening this, to adjust the test a little more; There's an option to run without a backup thread by setting the property -Dderby.nstest.backupRestore=false, but when you do this, the test runs into an ArrayIndexOutOfBounds error on creating the tester threads, which then confuses the workings of the main method so it won't finish cleanly. In that case, the Memcheck keeps going, so just to be on the safe side, I'll add a check for remaining test threads in the Memcheck. While I'm add it, I'll remove some commented-out settings that appear to be left behind from previous configuration attempts.
          Hide
          Myrna van Lunteren added a comment -

          I don't intend to do more work on this at this time.

          Show
          Myrna van Lunteren added a comment - I don't intend to do more work on this at this time.
          Hide
          Myrna van Lunteren added a comment - - edited

          Committed to trunk with revision 1300658 and backported to 10.8 with revision 1300682.

          Show
          Myrna van Lunteren added a comment - - edited Committed to trunk with revision 1300658 and backported to 10.8 with revision 1300682.
          Hide
          Kristian Waagan added a comment -

          I had a quick look at the patch, +1 on the cleanup.

          A few comments/questions:
          o the comments in setSmallConfig look a bit odd, did you intend to remove the public static int part?
          o missing space in '("FAIL: " + thread_id + " inserted " + rowsAdded + "rows");'?
          o where printouts are commented out, can they be deleted or does it make sense to control them with a debug flag (similar to BaseTestCase.println)?
          (the flag doesn't have to be tied to a property)
          o you say you want to stick with the indentation of the original code, but spaces crept in next to tabs in a few places

          Show
          Kristian Waagan added a comment - I had a quick look at the patch, +1 on the cleanup. A few comments/questions: o the comments in setSmallConfig look a bit odd, did you intend to remove the public static int part? o missing space in '("FAIL: " + thread_id + " inserted " + rowsAdded + "rows");'? o where printouts are commented out, can they be deleted or does it make sense to control them with a debug flag (similar to BaseTestCase.println)? (the flag doesn't have to be tied to a property) o you say you want to stick with the indentation of the original code, but spaces crept in next to tabs in a few places
          Hide
          Myrna van Lunteren added a comment -

          Note that for this patch, I kept the indentation to match the original code.

          Show
          Myrna van Lunteren added a comment - Note that for this patch, I kept the indentation to match the original code.
          Hide
          Myrna van Lunteren added a comment -

          Attaching a patch with a number of these minor improvements.

          The small nstest configuration now runs in about 20 minutes on my slow machine.
          I will run some more experiments and if all OK, I'll add a comment about it to the README.txt and commit.

          Show
          Myrna van Lunteren added a comment - Attaching a patch with a number of these minor improvements. The small nstest configuration now runs in about 20 minutes on my slow machine. I will run some more experiments and if all OK, I'll add a comment about it to the README.txt and commit.
          Hide
          Myrna van Lunteren added a comment -

          Also found two e.printStackTrace(); calls in DbUtil. One should do.

          Show
          Myrna van Lunteren added a comment - Also found two e.printStackTrace(); calls in DbUtil. One should do.

            People

            • Assignee:
              Myrna van Lunteren
              Reporter:
              Myrna van Lunteren
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development