[GEODE-6824] CI Failure: BackupIntegrationTest

• Type: Test
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.10.0
• Component/s: persistence
• Assignee: Jens Deppe
• Reporter: Jens Deppe
• Created: 31/May/19 20:55
• Updated: 26/Sep/19 18:05
• Resolved: 21/Jun/19 15:07

Description

On Windows 2016, this test fails with errors like this:

org.apache.geode.internal.cache.backup.BackupIntegrationTest > testIncrementalBackupAndRecover FAILED
    java.lang.AssertionError: Restore scripts [] expected:<1> but was:<0>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:834)
        at org.junit.Assert.assertEquals(Assert.java:645)
        at org.apache.geode.internal.cache.backup.BackupIntegrationTest.restoreBackup(BackupIntegrationTest.java:443)
        at org.apache.geode.internal.cache.backup.BackupIntegrationTest.testIncrementalBackupAndRecover(BackupIntegrationTest.java:235)

The logs contain more indicators of what's going wrong:

[warn 2019/05/31 10:08:47.953 GMT <BackupServiceThread1> tid=0xf5] Unable to delete temporary directory created during backup, C:\Users\geode\AppData\Local\Temp\backup_15592973278755095066745076642151
java.io.IOException: Unable to delete file: C:\Users\geode\AppData\Local\Temp\junit2122524286779777274\disk_Dir2\backupTemp_1559297327875\BACKUPdiskStore_2.crf
	at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2400)
	at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1721)
	at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1617)
	at org.apache.geode.internal.cache.backup.TemporaryBackupFiles.deleteDirectory(TemporaryBackupFiles.java:133)
	at org.apache.geode.internal.cache.backup.TemporaryBackupFiles.cleanupFiles(TemporaryBackupFiles.java:126)
	at org.apache.geode.internal.cache.backup.BackupTask.cleanup(BackupTask.java:183)
	at org.apache.geode.internal.cache.backup.BackupTask.doBackup(BackupTask.java:125)
	at org.apache.geode.internal.cache.backup.BackupTask.backup(BackupTask.java:82)
	at org.apache.geode.internal.cache.backup.BackupService.lambda$prepareBackup$0(BackupService.java:62)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Way under the covers, during a backup, we create hard links from the original file to a backup file (if hard linking fails then there is a fallback to simply copy the file).

My guess is that the semantics of hard links may have changed between Windows versions (which is why we're suddenly seeing this on Windows 2016).

Issue Links

• links to: GitHub Pull Request #3658