[DERBY-5738] SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE only returns to the OS pages in the last extent

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.2.1, 10.7.1.1, 10.8.2.2
• Component/s: Store
• Labels: derby_triage10_12
• Urgency: Urgent
• Issue & fix info: High Value Fix
• Bug behavior facts: Seen in production
• Assignee: Unassigned
• Reporter: Mike Matrigali
• Created: 02/May/12 23:09
• Updated: 21/Jan/15 21:58

Description

It looks like inplace compress only gives back space to the OS from the empty pages in the last extent of the file. If the defrag and purge phases are
run they can reclaim internal space in the form of free pages to be used for future inserts, but free pages before last extent are not returned to
OS.

Only current workaround is to use SYSCS_UTIL.SYSCS_COMPRESS_TABLE instead.

In filecontainer.java!compressContainer there is a comment with steps to be performed, but no code exists for step 5 and 6:

This is outline of the algorithm used in compressing the container.

Until a non free page is found loop, in each loop return to the OS
all space at the end of the container occupied by free pages, including
the allocation page itself if all of it's pages are free.

1) Find last 2 allocation pages in container (last if there is only one).
2) invalidate the allocation information cached by the container.
Without the cache no page can be gotten from the container. Pages
already in the page cache are not affected. Thus by latching the
allocPage and invalidating the allocation cache, this NTT blocks out
all page gets from this container until it commits.
3) the allocPage determines which pages can be released to the OS,
mark that in its data structure (the alloc extent). Mark the
contiguous block of nallocated/free pages at the end of the file
as unallocated. This change is associated with the NTT.
4) The NTT calls the OS to deallocate the space from the file. Note
that the system can handle being booted and asked to get an allocated
page which is past end of file, it just extends the file automatically.
5) If freeing all space on the alloc page, and there is more than one
alloc page, then free the alloc page - this requires an update to the
previous alloc page which the loop has kept latched also.
6) if the last alloc page was deleted, restart loop at #1

Issue Links

• is related to: DERBY-5234 Unable to insert data into table. Failed due be "ERROR XSDG0: Page Page(51919,Container(0, 1104)) could not be read from disk."
• is related to: DERBY-5876 Overhaul table compression code