Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5399

PagingFieldCollector is very slow with String fields

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.7, 6.0
    • Component/s: core/search
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      PagingFieldCollector (sort comparator) is significantly slower with string fields, because of how its "seen on a previous page" works: it calls compareDocToValue(int doc, T t) first to check this. (its the only user of this method)

      This is very slow with String, because no ordinals are used. so each document must lookup ord, then lookup bytes, then compare bytes.

      I think maybe we should replace this method with an 'after' slot, and just have compareDocToAfter or something.

      Otherwise we could use a hack-patch like the one i will upload (i did this just to test the performance, although tests do pass).

        Attachments

        1. LUCENE-5399.patch
          5 kB
          Robert Muir
        2. LUCENE-5399.patch
          28 kB
          Michael McCandless
        3. LUCENE-5399.patch
          30 kB
          Michael McCandless
        4. LUCENE-5399.patch
          30 kB
          Michael McCandless
        5. LUCENE-5399.patch
          57 kB
          Michael McCandless
        6. LUCENE-5399.patch
          71 kB
          Michael McCandless
        7. LUCENE-5399.patch
          71 kB
          Robert Muir
        8. LUCENE-5399.patch
          71 kB
          Robert Muir
        9. LUCENE-5399.patch
          140 kB
          Michael McCandless

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rcmuir Robert Muir
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: