[JENA-1516] Concurrency issues with ObjectFileStorage

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Jena 3.5.0, Jena 3.6.0
• Fix Version/s: Jena 3.7.0
• Component/s: TDB
• Assignee: Andy Seaborne
• Reporter: Andy Seaborne
• Created: 03/Apr/18 20:36
• Updated: 01/Aug/18 18:30
• Resolved: 05/Apr/18 09:09

Description

Original user@jena email report.

This is under high load. In the attached test case, at least 50 queries and 50 updates issues by 25 threads are necessary to get any sort of likelihood of a failure.

Failures of alloc-write are exasperated by the fact flag for checking is not volatile.

The database is not permanently corrupted, however, updates can be lost.

Errors include the ones below - this is not exhaustive.
Errors, generated with a v3.6.0 server:
Some are consequences of earlier errors.

rm -rf DB ; fuseki-basic --loc DB --update /ds

21:15:56 qtp826865256-16 ERROR BindingTDB           :: get1(?s)
java.nio.BufferOverflowException
	at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:214)
	at sun.nio.ch.IOUtil.read(IOUtil.java:200)
	at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:741)
	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727)
	at org.apache.jena.tdb.base.file.BufferChannelFile.read(BufferChannelFile.java:112)

21:15:56 qtp826865256-13 ERROR BindingTDB           :: get1(?p)
java.lang.IllegalArgumentException
	at java.nio.Buffer.position(Buffer.java:244)
	at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:216)
	at sun.nio.ch.IOUtil.read(IOUtil.java:200)
	at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:741)
	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727)
	at org.apache.jena.tdb.base.file.BufferChannelFile.read(BufferChannelFile.java:112)

21:15:56 qtp826865256-16 ERROR BindingTDB           :: get1(?s)
org.apache.jena.tdb.base.file.FileException: ObjectFileStorage.read[nodes](486)[filesize=2280][file.size()=2280]: Failed to read the length : got 0 bytes
	at org.apache.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:341)
	at org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57)
	at org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57)
	at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78)

21:15:56 qtp826865256-15 ERROR BindingTDB           :: get1(?o)
org.apache.jena.tdb.base.file.FileException: ObjectFileStorage.read[nodes](59)[filesize=2326][file.size()=2326]: Failed to read the length : got 0 bytes
	at org.apache.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:341)
	at org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57)
	at org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57)
	at org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57)
	at org.apache.jena.tdb.base.objectfile.ObjectFileWrapper.read(ObjectFileWrapper.java:57)
	at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78)

21:15:56 qtp826865256-13 ERROR BindingTDB           :: get1(?s)
java.nio.BufferOverflowException
	at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:214)
	at sun.nio.ch.IOUtil.read(IOUtil.java:200)
	at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:741)
	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727)
	at org.apache.jena.tdb.base.file.BufferChannelFile.read(BufferChannelFile.java:112)
	at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78)

21:15:56 qtp826865256-13 ERROR BindingTDB           :: get1(?s)
java.lang.IllegalArgumentException
	at java.nio.Buffer.position(Buffer.java:244)
	at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:216)
	at sun.nio.ch.IOUtil.read(IOUtil.java:200)
	at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:741)
	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727)
	at org.apache.jena.tdb.base.file.BufferChannelFile.read(BufferChannelFile.java:112)

Issue Links

• causes: JENA-1581 BufferOverflowException in TDB1
• links to: GitHub Pull Request #391