[SOLR-1623] Solr hangs (often throwing java.lang.OutOfMemoryError: PermGen space) when indexing many different field names

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Won't Fix
• Affects Version/s: 1.3, 1.4
• Component/s: update

Environment

Tomcat Version JVM Version JVM Vendor OS Name OS Version OS Architecture
Apache Tomcat/6.0 snapshot 1.6.0_13-b03 Sun Microsystems Inc. Linux 2.6.18-164.el5 amd64

and/or

Tomcat Version JVM Version JVM Vendor OS Name OS Version OS Architecture
Apache Tomcat/6.0.18 1.6.0_12-b04 Sun Microsystems Inc. Windows 2003 5.2 amd64

• Assignee: Unassigned
• Reporter: Laurent Chavet
• Created: 04/Dec/09 20:43
• Updated: 01/Oct/16 17:38
• Resolved: 01/Oct/16 17:38

Description

With the following fields in schema.xml:

<fields>
<field name="id" type="sint" indexed="true" stored="true" required="true" />
<dynamicField name="weight_*" type="sint" indexed="true" stored="true"/>
</fields>

Run the following code:

import java.util.ArrayList;
import java.util.List;
import org.apache.solr.client.solrj.SolrServer;
import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer;
import org.apache.solr.common.SolrInputDocument;

public static void main(String[] args) throws Exception {
SolrServer server;
try

{ server = new CommonsHttpSolrServer(args[0]); }

catch (Exception e)

{ System.err.println("can't creater server using: " + args[0] + " " + e.getMessage()); throw e; }

for (int i = 0; i < 1000; i++) {
List<SolrInputDocument> batchedDocs = new ArrayList<SolrInputDocument>();
for (int j = 0; j < 1000; j++)

{ SolrInputDocument doc = new SolrInputDocument(); doc.addField("id", i * 1000 + j); // hangs after 30 to 50 batches doc.addField("weight_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + Integer.toString(i) + "_" + Integer.toString(j), i * 1000 + j); // hangs after about 200 batches //doc.addField("weight_" + Integer.toString(i) + "_" + Integer.toString(j), i * 1000 + j); batchedDocs.add(doc); }

try

{ server.add(batchedDocs, true); System.err.println("Done with batch=" + i); // server.commit(); //doesn't change anything }

catch (Exception e)

{ System.err.println("batchId=" + i + " bad batch: " + e.getMessage()); throw e; }

}
}

And soon the client (sometime throws) and solr will freeze. sometime you can see: java.lang.OutOfMemoryError: PermGen space in the server logs