Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
When a db compaction is interrupted and restarted, it's possible for the most recent header written to the .compact.meta file to be buried under many GB from the end of file. For example, an interrupted compaction left the following files:
-rw-r--r-- 1 dbcore dbcore 47G Jun 27 04:29 /srv/db/shards/40000000-7fffffff/opendi/yellow-ng-loadbalancer.1461156255.couch.compact.data -rw-r--r-- 1 dbcore dbcore 12G Jun 27 05:20 /srv/db/shards/40000000-7fffffff/opendi/yellow-ng-loadbalancer.1461156255.couch.compact.meta
but
# grep -abo db_header yellow-ng-loadbalancer.1461156255.couch.compact.meta | tail -1 4364894251:db_header
which means the current algorithm must search through about 8GB of file before it gets to a header. I measured how long it takes, both on an SSD laptop and SSD server, getting similar numbers:
(dbcore@db3.testy012.cloudant.net)18> timer:tc(couch_file,test_find_header,["/srv/jdoane/yellow-ng-loadbalancer.1461156255.couch.compact.meta", default]). {328335338, ... (node1@127.0.0.1)27> timer:tc(couch_file,test_find_header,["/Users/jay/proj/ibm/sample-data/yellow-ng-loadbalancer.1461156255.couch.compact.meta", default]). {426650530, ...
which is between 328-427 seconds, or 19-25 MB/s.
One reason for this relative slowness is because the current algorithm performs a disk read for every block it searches:
https://github.com/apache/couchdb-couch/blob/master/src/couch_file.erl#L537-L539
We can improve the speed by loading a "chunk" of many blocks into memory with a single read operation, and then search each block in memory, trading off memory for speed. Ideally, the tradeoff can be made configurable, so that existing speed/memory behavior can be retained by default.
Attachments
Issue Links
- links to
