Lucene - Core
  1. Lucene - Core
  2. LUCENE-5048

NegativeArraySizeException caused by ff.addFields


    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.2, 4.3
    • Fix Version/s: 4.4, master
    • Component/s: modules/facet
    • Labels:
    • Environment:

      Not sure if this is applicable, but it's Gentoo Linux with java 1.7 on a dual intel xeon lga2011 with supermicro server motherboard and 256GB of ram

    • Lucene Fields:
      New, Patch Available


      I have a Server/Client software that I have created which has a server process that accepts connections from clients that transmit data about local connection information. This data is than buffered and a ThreadPoolExecutor runs to take the data and put it into a lucene index as well as a facet index. This works perfect for the lucene index, but the facet index randomly generates a NegativeArraySizeException. I cannot find any reason why the exception would be caused because lines with the same type of data do not throw it, then all of a sudden the exception is thrown, typically 4 of them in a row. I talked with mikemccand on IRC and he requested I submit this issue.

      After some discussion, he seems to think it's because some of the values I am using are rather large.

      Here is the exception...
      at java.lang.AbstractStringBuilder.<init>(
      at java.lang.StringBuilder.<init>(
      at org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.subSequence(
      at org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.hashCodeOfSerialized(
      at org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.stringHashCode(
      at org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.grow(
      at org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.addLabel(
      at org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.put(
      at org.apache.lucene.facet.index.FacetFields.addFields(
      at java.util.concurrent.ThreadPoolExecutor.runWorker(
      at java.util.concurrent.ThreadPoolExecutor$

      Here is an example data entry which appears when the exception occurs...
      Location: nj
      LocalPorts: [443]
      Connections: 1
      Times: [120]
      Timestamp: 2013-06-09T12:51:00.000-07:00
      States: ["Established"]

      And here is the the code stripped down to provide an example of how I am handling the facet/doc code.

      doc.add(new TextField("Location", ehost[0], Field.Store.YES));
      cats.add(new CategoryPath("Location", doc.get("Location")));
      doc.add(new TextField("LocalIP", (String) stat.get("LocalIP"), Field.Store.YES));
      cats.add(new CategoryPath("LocalIP", doc.get("LocalIP")));
      doc.add(new TextField("RemoteIP", (String) stat.get("RemoteIP"), Field.Store.YES));
      cats.add(new CategoryPath("RemoteIP", doc.get("RemoteIP")));
      doc.add(new TextField("LocalPorts", StringUtils.join(stat.get("LocalPorts"), ","), Field.Store.YES));
      cats.add(new CategoryPath("LocalPorts", doc.get("LocalPorts")));
      doc.add(new TextField("RemotePorts", StringUtils.join(stat.get("RemotePorts"), ","), Field.Store.YES));
      cats.add(new CategoryPath("RemotePorts", doc.get("RemotePorts")));
      doc.add(new LongField("Connections", (Long) stat.get("Connections"), Field.Store.YES));
      cats.add(new CategoryPath("Connections", doc.get("Connections")));
      doc.add(new TextField("Times", StringUtils.join(stat.get("Times"), ","), Field.Store.YES));
      cats.add(new CategoryPath("Times", doc.get("Times")));
      doc.add(new TextField("Timestamp", (String) stat.get("Timestamp"), Field.Store.YES));
      cats.add(new CategoryPath("Timestamp", doc.get("Timestamp")));
      doc.add(new TextField("States", StringUtils.join(stat.get("States"), ","), Field.Store.YES));
      cats.add(new CategoryPath("States", doc.get("States")));
      System.out.println("Location: "doc.get("Location")" LocalIP: "doc.get("LocalIP")" RemoteIP: "doc.get("RemoteIP")" LocalPorts: "doc.get("LocalPorts")" Connections: "doc.get("Connections")" Times: "doc.get("Times")" Timestamp: "doc.get("Timestamp")" States: "+doc.get("States"));
      if (cats.size()!=0)

      { FacetFields ff = new FacetFields(Main.twriter); ff.addFields(doc, cats); // <-- Exception occurs here randomly }
      1. LUCENE-5048.patch
        2 kB
        Michael McCandless
      2. LUCENE-5048.patch
        7 kB
        Shai Erera
      3. LUCENE-5048.patch
        9 kB
        Shai Erera
      4. LUCENE-5048.patch
        11 kB
        Shai Erera


        No work has yet been logged on this issue.


          • Assignee:
            Shai Erera
            Colton Jamieson
          • Votes:
            0 Vote for this issue
            3 Start watching this issue


            • Created: