OFBiz
  1. OFBiz
  2. OFBIZ-2259

Testing - Rollback database changes after each test-suite using the GenericDelegator

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: SVN trunk
    • Component/s: framework
    • Labels:
      None

      Description

      During testing have the delegator keep track of all changes made to the database and then roll them back at the end of each test suite. This will allow the same demo data to be reused across test suites without having to worry about what changes previous tests have made to the data and solves this problem regardless of the database being used.

      1. rollback.patch
        13 kB
        Scott Gray
      2. rollback.patch
        57 kB
        Scott Gray
      3. rollback.patch
        54 kB
        Scott Gray

        Activity

        Hide
        Scott Gray added a comment -

        Forgot to close, this was committed in r765057

        Show
        Scott Gray added a comment - Forgot to close, this was committed in r765057
        Hide
        Scott Gray added a comment -

        Sorry that patch had a few issues, here's a better version

        Show
        Scott Gray added a comment - Sorry that patch had a few issues, here's a better version
        Hide
        Scott Gray added a comment -

        Updated version of rollback functionality.

        David, the cloned delegator was a bit more work than I first thought and meant I had to change a few things in various places that could possibly have side effects so it would be great if you could take another look and let me know what you think.

        Also, there are currently 3 types of thing that aren't rolled back during testing:
        1. Data loaded through the entity-xml test tag, this is because EntitySaxReader also clones the delegator but I don't why it does that so I've hesitated to change how it works. I would like to resolve this one before committing though.
        2. Data stored during async service calls
        3. Changes to the SequenceValueItem table, I haven't looked but I'm guessing rolling that back might cause problems and it's not such a big deal anyway.

        Regards
        Scott

        Show
        Scott Gray added a comment - Updated version of rollback functionality. David, the cloned delegator was a bit more work than I first thought and meant I had to change a few things in various places that could possibly have side effects so it would be great if you could take another look and let me know what you think. Also, there are currently 3 types of thing that aren't rolled back during testing: 1. Data loaded through the entity-xml test tag, this is because EntitySaxReader also clones the delegator but I don't why it does that so I've hesitated to change how it works. I would like to resolve this one before committing though. 2. Data stored during async service calls 3. Changes to the SequenceValueItem table, I haven't looked but I'm guessing rolling that back might cause problems and it's not such a big deal anyway. Regards Scott
        Hide
        Jacques Le Roux added a comment -

        Yes good idea !

        Show
        Jacques Le Roux added a comment - Yes good idea !
        Hide
        Scott Gray added a comment -

        Hi Jacopo

        I think it is possible but we couldn't use the new data in regular demo data files due to any updates performed that could break existing unit tests. We could definitely use the generated data within test suites though:
        <test-case case-name="load-service-test-data">
        <entity-xml action="load" entity-xml-url="component://service/testdef/data/ServiceTestData.xml"/>
        </test-case>
        Sounds pretty cool and should be fairly easy to implement, I'll add it to my todo list

        Show
        Scott Gray added a comment - Hi Jacopo I think it is possible but we couldn't use the new data in regular demo data files due to any updates performed that could break existing unit tests. We could definitely use the generated data within test suites though: <test-case case-name="load-service-test-data"> <entity-xml action="load" entity-xml-url="component://service/testdef/data/ServiceTestData.xml"/> </test-case> Sounds pretty cool and should be fairly easy to implement, I'll add it to my todo list
        Hide
        Jacopo Cappellato added a comment -

        Scott,

        looks like a great enhancements, thanks for working at it.
        Do you think we could use this work also as a way to prepare demo data for a test?
        What I have in mind is something like:
        1) I start ofbiz and when I am ready to prepare data I activate the "test mode"
        2) I prepare all the data I need
        3) instead of just rolling back everything we export all the changes recorded by the delegator in an xml file

        The data could then be used to prepare the starting scenario for a use case/test.

        Do you think it could work?

        Jacopo

        Show
        Jacopo Cappellato added a comment - Scott, looks like a great enhancements, thanks for working at it. Do you think we could use this work also as a way to prepare demo data for a test? What I have in mind is something like: 1) I start ofbiz and when I am ready to prepare data I activate the "test mode" 2) I prepare all the data I need 3) instead of just rolling back everything we export all the changes recorded by the delegator in an xml file The data could then be used to prepare the starting scenario for a use case/test. Do you think it could work? Jacopo
        Hide
        Scott Gray added a comment -

        Thanks for the feedback David, those are great points and I'll get them addressed as soon as I get a spare moment.

        Show
        Scott Gray added a comment - Thanks for the feedback David, those are great points and I'll get them addressed as soon as I get a spare moment.
        Hide
        David E. Jones added a comment -

        Thinking about this more I like the idea of a sort of cloned delegator. In other words instead of doing something like:

        modelSuite.getDelegator().setTestMode(true);

        we would do something like:

        GenericDelegator testDelegator = modelSuite.getDelegator().makeTestDelegator();

        and the later on near the end instead of:

        modelSuite.getDelegator().performUndo();
        modelSuite.getDelegator().setTestMode(false);

        maybe something like:

        testDelegator.rollback();

        Show
        David E. Jones added a comment - Thinking about this more I like the idea of a sort of cloned delegator. In other words instead of doing something like: modelSuite.getDelegator().setTestMode(true); we would do something like: GenericDelegator testDelegator = modelSuite.getDelegator().makeTestDelegator(); and the later on near the end instead of: modelSuite.getDelegator().performUndo(); modelSuite.getDelegator().setTestMode(false); maybe something like: testDelegator.rollback();
        Hide
        David E. Jones added a comment -

        This looks pretty good in general.

        One small issue: it is not thread safe. At some point we may decide that running test suites one at a time is too slow. In other words, chances are running a few at a time will result in an over all performance increase compared to running them in serial.

        One way we could possibly do this is to clone the delegator and use one instance per test suite. That's the first that comes to mind anyway, there are probably better options... A ThreadLocal variable is something I always consider to be a bit of a hack, but they do have their place. In this case that might not work as a test case could have multiple threads (not sure if any do...).

        Show
        David E. Jones added a comment - This looks pretty good in general. One small issue: it is not thread safe. At some point we may decide that running test suites one at a time is too slow. In other words, chances are running a few at a time will result in an over all performance increase compared to running them in serial. One way we could possibly do this is to clone the delegator and use one instance per test suite. That's the first that comes to mind anyway, there are probably better options... A ThreadLocal variable is something I always consider to be a bit of a hack, but they do have their place. In this case that might not work as a test case could have multiple threads (not sure if any do...).
        Hide
        Scott Gray added a comment -

        It would be great to get some feedback on this before I commit it in the next couple of days.

        In terms of performance the rollback adds about 15-20% to the time taken to perform all tests which on my machine equates to about 20-30 seconds

        Show
        Scott Gray added a comment - It would be great to get some feedback on this before I commit it in the next couple of days. In terms of performance the rollback adds about 15-20% to the time taken to perform all tests which on my machine equates to about 20-30 seconds

          People

          • Assignee:
            Scott Gray
            Reporter:
            Scott Gray
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development