Derby
  1. Derby
  2. DERBY-5213

Write tests to verify the interaction of TRUNCATE TABLE and online backup

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.9.1.0
    • Fix Version/s: 10.10.1.1
    • Component/s: SQL, Store
    • Labels:
      None
    • Issue & fix info:
      Newcomer

      Description

      An uncommitted TRUNCATE TABLE command does not block online backup. We should verify that the online and backed up databases are both in a consistent state. At a minimum, we should test the following:

      o uncommitted truncate table followed by online backup and then access the backup copy and access the table. should see the old data.
      o uncommitred truncate table, followed by online backup that keeps logs,
      then commit the truncate, and then access the table in the backup.

      For more information, please see this email thread: http://old.nabble.com/truncating-a-table-vs-online-backup-to31524933.html#a31524933

      1. DERBY_5213.diff_1
        15 kB
        Myrna van Lunteren
      2. DERBY-5213.diff_2
        11 kB
        Myrna van Lunteren
      3. DERBY-5213.diff_3
        8 kB
        Myrna van Lunteren

        Activity

        Hide
        Myrna van Lunteren added a comment -

        I'm going to make a stab at writing a test for this.

        Show
        Myrna van Lunteren added a comment - I'm going to make a stab at writing a test for this.
        Hide
        Myrna van Lunteren added a comment -

        Attaching a patch which does mostly precisely what was suggested in the email exchange, and this patch is ready for review.

        I think there are some further tests that can be added, for instance, there's some discussion in the reference guide about using blocking or not blocking behavior of SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE and SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE_NOWAIT.

        Perhaps also a test that tries to do a rollback of the truncate table.

        Show
        Myrna van Lunteren added a comment - Attaching a patch which does mostly precisely what was suggested in the email exchange, and this patch is ready for review. I think there are some further tests that can be added, for instance, there's some discussion in the reference guide about using blocking or not blocking behavior of SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE and SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE_NOWAIT. Perhaps also a test that tries to do a rollback of the truncate table.
        Hide
        Myrna van Lunteren added a comment -

        oops, just realized I forgot to undo the changes to lang._Suite in my initial patch for this - I commented out a bunch of tests for a mini-trial run.
        That's of course not intended for commit.

        Show
        Myrna van Lunteren added a comment - oops, just realized I forgot to undo the changes to lang._Suite in my initial patch for this - I commented out a bunch of tests for a mini-trial run. That's of course not intended for commit.
        Hide
        Myrna van Lunteren added a comment -

        I could use a review of this preliminary patch.

        Show
        Myrna van Lunteren added a comment - I could use a review of this preliminary patch.
        Hide
        Rick Hillegas added a comment -

        Hi Myrna,

        It looks to me as though the test tackles both cases described on this JIRA. I'm a little confused by the queries which are supposed to check that the table has 1000 rows and 0 rows. It seems to me that the queries are only looking at the last 2 rows in the table and are not really verifying what they claim to. Maybe "select count from truncable" would be a better query. Thanks.

        Show
        Rick Hillegas added a comment - Hi Myrna, It looks to me as though the test tackles both cases described on this JIRA. I'm a little confused by the queries which are supposed to check that the table has 1000 rows and 0 rows. It seems to me that the queries are only looking at the last 2 rows in the table and are not really verifying what they claim to. Maybe "select count from truncable" would be a better query. Thanks.
        Hide
        Mike Matrigali added a comment -

        the tests in the preliminary patch look good. This is just from a reading of the code, did not compile/run it.

        Some nits:
        o would be nice to have hdr comment on testUncommittedTruncateBackupEnableLog
        o assertDirectoryDeleted() seems like something that should be in a test utility that could be
        used in all tests rather than specific tests.

        stuff that should not hold up a commit:
        o An interesting test would be to have truncate table happen "during the backup". This is a lot harder to write as timing is very
        critical. Spent some time thinking about this but could not come up with a user level code way to test this for sure. Could
        hack the actual code to help testing but that is not a great idea.
        o could add tests of freeze and unfreeze with respect to truncate.

        Show
        Mike Matrigali added a comment - the tests in the preliminary patch look good. This is just from a reading of the code, did not compile/run it. Some nits: o would be nice to have hdr comment on testUncommittedTruncateBackupEnableLog o assertDirectoryDeleted() seems like something that should be in a test utility that could be used in all tests rather than specific tests. stuff that should not hold up a commit: o An interesting test would be to have truncate table happen "during the backup". This is a lot harder to write as timing is very critical. Spent some time thinking about this but could not come up with a user level code way to test this for sure. Could hack the actual code to help testing but that is not a great idea. o could add tests of freeze and unfreeze with respect to truncate.
        Hide
        Myrna van Lunteren added a comment -

        Thank you for the reviews Rick and Mike.

        Rick - you are right of course. For some reason I just didn't want to do a select count, but it's the more logical query, so I changed it.
        Mike - I added a comment describing the second fixture, and there was already an assertDirectoryDeleted in org.apache.derbyTesting.junit.BaseTestCase, so I removed the local method.

        I'm running regressions now, and assuming they go well, I will commit this patch; then I'll start working on the other testing suggestions.

        Show
        Myrna van Lunteren added a comment - Thank you for the reviews Rick and Mike. Rick - you are right of course. For some reason I just didn't want to do a select count, but it's the more logical query, so I changed it. Mike - I added a comment describing the second fixture, and there was already an assertDirectoryDeleted in org.apache.derbyTesting.junit.BaseTestCase, so I removed the local method. I'm running regressions now, and assuming they go well, I will commit this patch; then I'll start working on the other testing suggestions.
        Hide
        Myrna van Lunteren added a comment -

        regression tests went fine, so I committed the _2.diff patch with revision 1380779 (http://svn.apache.org/viewvc?view=revision&revision=1380779)

        Show
        Myrna van Lunteren added a comment - regression tests went fine, so I committed the _2.diff patch with revision 1380779 ( http://svn.apache.org/viewvc?view=revision&revision=1380779 )
        Show
        Knut Anders Hatlen added a comment - There were some failures in the nightly tests after the new test went in: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/vista-64/1381230-suitesAll_diff.txt http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/sol/1381654-suitesAll_diff.txt http://dbtg.foundry.sun.com/derby/test/Daily/javaME/testing/testlog/ubuntu/1381230-suitesAll_diff.txt
        Hide
        Myrna van Lunteren added a comment - - edited

        Thanks for noticing those failures Knut.
        I'll take a look at it on Monday. I've disabled the test (commented out in lang._Suite) for now with revision: http://svn.apache.org/viewvc?view=revision&revision=1382114.

        Show
        Myrna van Lunteren added a comment - - edited Thanks for noticing those failures Knut. I'll take a look at it on Monday. I've disabled the test (commented out in lang._Suite) for now with revision: http://svn.apache.org/viewvc?view=revision&revision=1382114 .
        Hide
        Myrna van Lunteren added a comment -

        I've taken a quick look, and it seems to me the problem was that the sequence of the tests was important - one of the tests did a truncate table and committed it.
        As that wasn't my intention, I've removed the decorateSQL and put the relevant code into the setUp() and dropped the table in a teardown() method.

        Show
        Myrna van Lunteren added a comment - I've taken a quick look, and it seems to me the problem was that the sequence of the tests was important - one of the tests did a truncate table and committed it. As that wasn't my intention, I've removed the decorateSQL and put the relevant code into the setUp() and dropped the table in a teardown() method.
        Hide
        Myrna van Lunteren added a comment -

        Attaching a patch which

        • adds a third fixture, which tests an uncommitted truncate table followed by freeze/copy/unfreeze
        • removes the decorateSQL method and moves the creation and population of the truncable table to setUp(), and drops the table in teardown()
        • reinstates the test in lang._Suite

        I will commit this shortly.

        Show
        Myrna van Lunteren added a comment - Attaching a patch which adds a third fixture, which tests an uncommitted truncate table followed by freeze/copy/unfreeze removes the decorateSQL method and moves the creation and population of the truncable table to setUp(), and drops the table in teardown() reinstates the test in lang._Suite I will commit this shortly.
        Hide
        Myrna van Lunteren added a comment -

        Committed with revision http://svn.apache.org/viewvc?view=revision&revision=1383677.
        Looks like tests have passed ok. Resolving.

        Show
        Myrna van Lunteren added a comment - Committed with revision http://svn.apache.org/viewvc?view=revision&revision=1383677 . Looks like tests have passed ok. Resolving.

          People

          • Assignee:
            Myrna van Lunteren
            Reporter:
            Rick Hillegas
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development