Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-3298

FST has hard limit max size of 2.1 GB

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 4.2, 6.0
    • core/FSTs
    • None
    • New

    Description

      The FST uses a single contiguous byte[] under the hood, which in java is indexed by int so we cannot grow this over Integer.MAX_VALUE. It also internally encodes references to this array as vInt.

      We could switch this to a paged byte[] and make the far larger.

      But I think this is low priority... I'm not going to work on it any time soon.

      Attachments

        1. LUCENE-3298.patch
          59 kB
          Michael McCandless
        2. LUCENE-3298.patch
          38 kB
          Michael McCandless
        3. LUCENE-3298.patch
          3 kB
          Michael McCandless
        4. LUCENE-3298.patch
          30 kB
          James Dyer

        Issue Links

          Activity

            People

              mikemccand Michael McCandless
              mikemccand Michael McCandless
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: