Details

    • Type: Wish Wish
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.4
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: core/search
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      I propose adding LATENT FieldSelectorResult

      this would be similar to LAZY_LOAD except that it would NEVER cache the stored value

      This will be useful for very large fields that should always go direct to disk (because they will take so much memory)
      when caching Documents returned from a Searcher, the large field may be initially requested as LAZY_LOAD, however once someone reads this field, it will then get locked into memory. if this Document (and others like it) are cached, it can start to use a very large amount of memory for these fields

      Contract for FieldSelectorResult.LATENT should be that it will always be pulled direct from the IndexInput and never be persisted in memory as part of a Fieldable

      I could prepare a patch if desired

      1. LUCENE-1810.patch
        11 kB
        Grant Ingersoll
      2. LUCENE-1810.patch
        16 kB
        Tim Smith

        Activity

        Hide
        Grant Ingersoll added a comment -

        Patch would be great.

        Show
        Grant Ingersoll added a comment - Patch would be great.
        Hide
        Tim Smith added a comment -

        Attaching patch to provide FieldSelectorResult.LATENT

        this patch hooks in to add a "cacheResult" flag to LazyField

        LOAD_LAZY sets this flag to true
        LATENT sets this flag to false

        added test that verifies stringValue() returned from LAZY_LOAD is always the same, and stringValue() returned from LATENT is always different (reloaded each time)

        Show
        Tim Smith added a comment - Attaching patch to provide FieldSelectorResult.LATENT this patch hooks in to add a "cacheResult" flag to LazyField LOAD_LAZY sets this flag to true LATENT sets this flag to false added test that verifies stringValue() returned from LAZY_LOAD is always the same, and stringValue() returned from LATENT is always different (reloaded each time)
        Hide
        Grant Ingersoll added a comment -

        Updates to trunk. I intend to commit today or tomorrow.

        Show
        Grant Ingersoll added a comment - Updates to trunk. I intend to commit today or tomorrow.
        Hide
        Grant Ingersoll added a comment -

        Committed revision 959657 on trunk.

        Committed revision 959674 on 3.x

        Show
        Grant Ingersoll added a comment - Committed revision 959657 on trunk. Committed revision 959674 on 3.x
        Hide
        Grant Ingersoll added a comment -

        Bulk close for 3.1

        Show
        Grant Ingersoll added a comment - Bulk close for 3.1

          People

          • Assignee:
            Grant Ingersoll
            Reporter:
            Tim Smith
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development