Ahmet: thanks for raising this issue, and for your patch.

you didn't mention it in jira, but in your linked email you suggested that the method being tested (docListToSolrDocumentList) isn't actually doing what it's suppose to...

Regardless of Set<String> fields parameter, SolrPluginUtils#docListToSolrDocumentList method loads all of the stored fields. Shouldn't it just load the fields given in the set? Should I file a jira ticket?

Digging deeper, i found two problems...

• docListToSolrDocumentList is delegating the field list to SolrIndexSearcher.doc, which means those fields are read immediately, but the rest of the fields will be lazy loaded
• docListToSolrDocumentList is calling DocumentBuilder.loadStoredFields which loads all of the fields – regardless of which field list was passed to docListToSolrDocumentList
• docListToSolrDocumentList has some really bizare code in it, preventing stored fields from being returned if they are the result of a copyField.

Since there are no other usages of DocumentBuilder.loadStoredFields in Solr, it's not clear to me what the intended point of it's crazy logic was

additions to Ahmet's patch showing the additional problems found.

given that neither of these two methods are used anywhere in Solr, and they are so clearly broken i suggest we deprecate them in 3.6 as "completely broken" and remove in trunk.

barring objections i'm going to...

• remove these methods in trunk
• mark then "@deprecated apparently this method has behaved as documented for a very long time, and is unneeded in Solr, so it is deprecated and will be removed in Solr 4" in 3x

...tomorrow.

remove these methods in trunk

+1

I think this was used long ago before the javabin format took over serializing DocList

Oy...

so i started down the road of removing all this from trunk, and then realized i somehow overlooked the fact that clustering is using docListToSolrDocumentList

so then i started trying to fix clustering to have the logic it needed, and then realized that the code i wound up with actually implemented the documented contract of docListToSolrDocumentList in a way that made the new tests in this issue pass.

So this new patch:

• fixes docListToSolrDocumentList
• fixes the test for docListToSolrDocumentList
• removes DocumentBuilder.loadStoredFields

my new plan is to commit as is to trunk, and backport all but the DocumentBuilder.java changes to 3x (i'll just deprecate loadStoredFields in 3x)

comments?

r1303977 - trunk fix of SolrPluginUtils & test, removal of loadStoredFields method

r1304001 - backport to 3x, but leave loadStoredFields in place with a scary freaking deprecation warning.

Ahmet: thanks for your report and especially for your test case