[COUCHDB-852] CouchDB Windows - View building crashing fatally, unable to resume view building

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0
• Component/s: Database Core, JavaScript View Server

Environment

Windows 7 - 32bit - Erlang R14A (erts-5.8) [source] [smp:4:4] [rq:4] [async-threads:0]

• Skill Level: Regular Contributors Level (Easy to Medium)
• Assignee: Unassigned
• Reporter: Lim Yue Chuan
• Created: 11/Aug/10 04:23
• Updated: 21/Nov/10 19:39
• Resolved: 21/Nov/10 19:37

Description

I have a DB with about 270,000 documents, inserted via a ruby script. Documents inserted fine, I get a working list of documents in the _all_docs view, can open an individual document up, etc.

Tried to generate the appropriate views for the database by hitting a doc via futon, memory size/cpu usage stayed sane (35mb, 5-40% on a i5-750) for the first 180k documents. Then ballooned up to over 800mb and the view build eventually dies.

[info] [<0.151.0>] checkpointing view update at seq 181904 for gsc_lt2 _design/mip

Crash dump was written to: erl_crash.dump
eheap_alloc: Cannot allocate 373662860 bytes of memory (of type "old_heap").

(Crash log for the couch/io installer, Mark Hammond's installer dies with a similar error message, but eheap_alloc reports "of type "heap"" instead)

At which point CouchDB completely dies (expected, just mentioning it)

Restarting CouchDB and trying to resume the view build fails fatally as well, with CouchDB reporting enomem errors in mochiweb. At this point, the already built view indexes appear to be totally unusable, I am unable to clean/compact the views, and I cannot find any way of restarting the build process other then by deleting the file holding the view (the md5sum-named file).

Tested on two different harddisks, using both Windows installers (Mark Hammond + couch.io). Also further tested on request of benoitc on a vmware instance running ubuntu (NOT able to replicate the issue, view generation completes fine on the linux VM). Seems to be a Windows issue. VM instance had 1GB allocated to it, Machine has 4gb of RAM. Disk space on both tested harddisks remained > 20GB free at all times.

For comparison purposes, memory usage on VM was stable at 5% of system RAM (5% of 1GB) throughout the entire view generation.

The appropriate crash logs are:
erl_crash.7z - for the first crash.
couchdb.7z - containing two files:
couch - start-firstcrash.log - CouchDB log files from startup, through view generation, till the first crash
couch - start-secondcrash.log - CouchDB log files from startup, through view generation + first crash, then requerying views, till second (mochiweb) crash.

Issue Links

• blocks: COUCHDB-897 Couchdb crashes on Windows when a database file exceeds 4 GB
• duplicates: COUCHDB-897 Couchdb crashes on Windows when a database file exceeds 4 GB
• is cloned by: COUCHDB-897 Couchdb crashes on Windows when a database file exceeds 4 GB