Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Won't Fix
-
1.1
-
None
-
None
Description
When using external view servers, such as couchjs for example, if we trigger "os_process_limit" (or more) index updates simultaneously, we can reach a deadlock case.
The issue is that each index updater will acquire a couchjs (os_proc record) process from couch_query_servers to apply the map function against each document. After the indexer finishes, it will release (return to couch_query_servers) the couchjs process used for the maps.
However, while the indexer is writing to the btrees, if the reduce function is a JavaScript function (or any other language other than Erlang or a builtin reduce function), the function is called often to reduce key-values - this results in the view updater process to ask couch_query_servers for a another couchjs process (this is done on every reduce function call) - however "os_process_limit" couchjs processes are already allocated to "os_process_limit" indexers for the mapping.
The solution here would be to have the index updater to preallocate a couchjs process for the reduces and release it when it finishes (like it's done for the maps).
This only happens if the number of changes to index is greater than 500 (the work queue sizes).