[COUCHDB-2425] Exceptions in couch_proc_manager:new_proc/1 leave dangling couch_query_servers

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: JavaScript View Server
• Assignee: Unassigned
• Reporter: Mike Wallace
• Created: 31/Oct/14 19:34
• Updated: 27/Oct/15 21:51
• Resolved: 23/Jan/15 16:07

Description

If an exception is thrown in the try blocks in either of the couch_proc_manager:new_proc/1 clauses [1] [2] then a spawn_error is returned which is handled by a specific handle_info clause [3] which does not reply to the calling process. When the caller is couch_query_servers:get_os_process/1 [4] there is an infinite timeout so that process will hang around until either the node reboots or someone intervenes.

The user-facing symptom is an entry in _active_tasks that makes no progress and never goes away.

The easiest way to reproduce this is to create a new view and then patch the code to force an exception in the appropriate place, e.g.: (assuming a live dev/run instance):

1. Create DB, add a ddoc and a doc:

$ curl -X PUT http://localhost:15984/kitteh
{"ok":true}
$ curl -X POST http://localhost:15984/kitteh -d '{"_id":"_design/view", "views": {"test": {"map": "function(doc) { emit(doc.id, 1); }"}}}' -H 'Content-Type: application/json'
{"ok":true,"id":"_design/view","rev":"1-ef10f980522d4c8e691e3c26d4c3fac5"}
$ curl -X PUT http://localhost:15984/kitteh/ohai -d '{}'
{"ok":true,"id":"ohai","rev":"1-967a00dff5e02add41819138abb3284d"}

2. Apply https://gist.github.com/mikewallace1979/79f823da25f6a78e2725 then re-run make and dev/run.

3. Attempt to query the view:

$ curl -X GET http://localhost:15984/kitteh/_design/view/_view/test
{"error":"timeout","reason":"The request could not be processed in a reasonable amount of time."}

4. Observe index build tasks which never go away:

$ curl -X GET http://localhost:15986/_active_tasks
[{"pid":"<0.2403.0>","changes_done":0,"database":"shards/40000000-5fffffff/kitteh.1414779043","design_document":"_design/view","progress":0,"started_on":1414779439,"total_changes":1,"type":"indexer","updated_on":1414779439},{"pid":"<0.2444.0>","changes_done":0,"database":"shards/00000000-1fffffff/kitteh.1414779043","design_document":"_design/view","progress":0,"started_on":1414779439,"total_changes":1,"type":"indexer","updated_on":1414779439}]

[1] https://github.com/apache/couchdb-couch/blob/master/src/couch_query_servers.erl#L354

[2] https://github.com/apache/couchdb-couch/blob/master/src/couch_proc_manager.erl#L368-L384

[3] https://github.com/apache/couchdb-couch/blob/master/src/couch_proc_manager.erl#L239-L245

[4] https://github.com/apache/couchdb-couch/blob/master/src/couch_query_servers.erl#L354