The results are slightly surprising.
I made up an index, and each document contained 4 random numbers between 1 and 500,000
This is not the distribution one would expect to see in a real index. but we can still learn much.
The synthetic index:
number of segments = 5
number of unique facet terms = 490903
filterCache max size = 1,000,000 entries (more than enough)
JVM=1.5.0_09 -server -Xmx200M
System=WinXP, 3GHz P4, hyperthreaded, 1GB dual channel RAM
facet type = facet.field, facet.sort=true, facet.limit=10
maximum df of any term = 15
warming times were not included... queries were run many times and the lowest time recorded.
Number of documents that match test "base" queries (for example, base query #1 matches 175K docs):
WITHOUT PATCH (milliseconds to facet each base query):
1578, 1578, 1547, 1485, 1484,1422
WITH PATCH (min df comparison w/ term df, minDfFilterCache=0) (all field cache)
984, 1203, 1391, 1437, 1484, 1420
WITH PATCH (min df comp, minDfFilterCache=30) (no fieldCache at all)
1406, 2344, 3125, 3015, 3172, 3172
CONCLUSION1: min df comparison increases faceting speed 60% when the base query matches many documents. With a real term distribution, this could be even greater.
CONCLUSION2: opting to not use the fieldCache for smaller df terms can save a lot of memory, but it hurts performance up to 200% for our non-optimized index.
CONCLUSION3: using the field cache less can significantly speed up warming time (times not shown, but a full warming of the fieldCache took 33 sec)
======== now the same index, but optimized ===========
WITH PATCH (optimized, min df comparison w/ term df, minDfFilterCache=0) (all field cache)
172, 312, 485, 578, 610, 656
WITH PATCH (optimized, min df comp, minDfFilterCache=30) (no fieldCache at all)
265, 344, 422, 468, 500, 484
CONCLUSION3: An optimized index increased performance 200-500%
CONCLUSION4: The fact that an all-fieldcache option was significantly faster on an optimized probably cannot totally be explained by accurate dfs (no deleted documents to inflate the term df values), means that just iterating over the terms is much faster in an optimized index (a potential Lucene area to look into)