Issue Details (XML | Word | Printable)

Key: DERBY-96
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Suresh Thalamati
Reporter: Suresh Thalamati
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Derby

partial log record writes that occur because of out-of order writes need to be handled by recovery.

Created: 09/Dec/04 08:11 PM   Updated: 01/Jul/09 12:34 AM
Return to search
Component/s: Store
Affects Version/s: 10.0.2.1
Fix Version/s: 10.1.1.0

Time Tracking:
Not Specified

Resolution Date: 28/May/05 09:07 AM


 Description  « Hide
Incomplete log record write that occurs because of
an out of order partial writes gets recognized as complete during
recovery if the first sector and last sector happens to get written.
 Current system recognizes incompletely written log records by checking
the length of the record that is stored in the beginning and end.
 Format the log records are written to disk is:

  +----------+-------------+------------------+

  | length | LOG RECORD | length |

  +----------+-------------+------------------+


This mechanism works fine if sectors are written in sequential manner or
log record size is less than 2 sectors. I believe on SCSI types disks
order is not necessarily sequential, SCSI disk drives may sometimes do a
reordering of the sectors to optimize the performance. If a log record
that spans multiple disk sectors is being written to SCISI type of
devices, it is possible that first and last sector written before the
crash; If this occurs recovery system will incorrectly interpret the
log records was completely written and replay the record. This could
lead to recovery errors or data corruption.
-


This problem also will not occur if a disk drive has write cache with a
battery backup which will make sure I/O request will complete.

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Repository Revision Date User Message
ASF #164994 Wed Apr 27 15:41:45 UTC 2005 mikem committing upgrade change related to DERBY-96 for suresh.thalamati.suresh.thalamati@gmail.com

Attached is the new patch with the suggested changes to make
softupgrade correctly with the transaction log
checksum feature in 10.1 Added checkVersion() method to log factory it
self, becuase that is where
the version numbers are read from from the log control file , but did
not export the call it to the
rawstore factory as it is not needed now. (This can be done easlily
when there is a need for upgrade
checks in the other store modules..)
Files Changed
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/log/LogAccessFile.java
MODIFY /incubator/derby/code/trunk/java/engine/org/apache/derby/iapi/store/raw/RawStoreFactory.java

Repository Revision Date User Message
ASF #169737 Thu May 12 00:09:32 UTC 2005 mikem committing tests for DERBY-96 for suresh.thalamati@gmail.com

Attached are some functional tests to test the transaction log checksum feature (DERBY-96).
Log corruption is simulated using a proxy storage factory that allows corruption of
the log write request before being writing to the disk. CorruptDiskStorage factory by default forwards
all the request to the underlying disk storage factory except when corruption flags are enabled.

Recovery tests need to boot the same database many times and have to use the
different Subprotocol to enable the corruption instead of the default protocol. This seems to be
possible only by adding a new tests suite in the current test frame work. Added a new suite
called "storerecovery" , may be all future recovery tests can be added to this suite.
Files Changed
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/util/corruptio/CorruptBaseStorageFactory.java
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/LogChecksumRecovery_derby.properties
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/LogChecksumSetup_app.properties
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/LogChecksumRecovery1_app.properties
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/util/corruptio/CorruptFile.java
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/LogChecksumRecovery_app.properties
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/util/corruptio/CorruptibleIo.java
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/util/corruptio
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/LogChecksumSetup.java
MODIFY /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/copyfiles.ant
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/LogChecksumRecovery1.java
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/storerecovery.properties
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/util/corruptio/CorruptDiskStorageFactory.java
MODIFY /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/storeall.properties
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/storerecovery.runall
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/LogChecksumRecovery1.out
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/LogChecksumRecovery.out
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/LogChecksumSetup.out
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/LogChecksumRecovery.java
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/LogChecksumSetup_derby.properties
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/LogChecksumRecovery1_derby.properties
ADD /incubator/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/util/corruptio/CorruptRandomAccessFile.java