Solr
  1. Solr
  2. SOLR-1623

Solr hangs (often throwing java.lang.OutOfMemoryError: PermGen space) when indexing many different field names

    Details

    • Type: Bug Bug
    • Status: Reopened
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: 1.3, 1.4
    • Fix Version/s: None
    • Component/s: update
    • Labels:
      None
    • Environment:

      Description

      With the following fields in schema.xml:

      <fields>
      <field name="id" type="sint" indexed="true" stored="true" required="true" />
      <dynamicField name="weight_*" type="sint" indexed="true" stored="true"/>
      </fields>

      Run the following code:

      import java.util.ArrayList;
      import java.util.List;
      import org.apache.solr.client.solrj.SolrServer;
      import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer;
      import org.apache.solr.common.SolrInputDocument;

      public static void main(String[] args) throws Exception {
      SolrServer server;
      try

      { server = new CommonsHttpSolrServer(args[0]); }

      catch (Exception e)

      { System.err.println("can't creater server using: " + args[0] + " " + e.getMessage()); throw e; }

      for (int i = 0; i < 1000; i++) {
      List<SolrInputDocument> batchedDocs = new ArrayList<SolrInputDocument>();
      for (int j = 0; j < 1000; j++)

      { SolrInputDocument doc = new SolrInputDocument(); doc.addField("id", i * 1000 + j); // hangs after 30 to 50 batches doc.addField("weight_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + Integer.toString(i) + "_" + Integer.toString(j), i * 1000 + j); // hangs after about 200 batches //doc.addField("weight_" + Integer.toString(i) + "_" + Integer.toString(j), i * 1000 + j); batchedDocs.add(doc); }

      try

      { server.add(batchedDocs, true); System.err.println("Done with batch=" + i); // server.commit(); //doesn't change anything }

      catch (Exception e)

      { System.err.println("batchId=" + i + " bad batch: " + e.getMessage()); throw e; }

      }
      }

      And soon the client (sometime throws) and solr will freeze. sometime you can see: java.lang.OutOfMemoryError: PermGen space in the server logs

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Laurent Chavet
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development