Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5399

PagingFieldCollector is very slow with String fields

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.7, 6.0
    • core/search
    • None
    • New

    Description

      PagingFieldCollector (sort comparator) is significantly slower with string fields, because of how its "seen on a previous page" works: it calls compareDocToValue(int doc, T t) first to check this. (its the only user of this method)

      This is very slow with String, because no ordinals are used. so each document must lookup ord, then lookup bytes, then compare bytes.

      I think maybe we should replace this method with an 'after' slot, and just have compareDocToAfter or something.

      Otherwise we could use a hack-patch like the one i will upload (i did this just to test the performance, although tests do pass).

      Attachments

        1. LUCENE-5399.patch
          140 kB
          Michael McCandless
        2. LUCENE-5399.patch
          71 kB
          Robert Muir
        3. LUCENE-5399.patch
          71 kB
          Robert Muir
        4. LUCENE-5399.patch
          71 kB
          Michael McCandless
        5. LUCENE-5399.patch
          57 kB
          Michael McCandless
        6. LUCENE-5399.patch
          30 kB
          Michael McCandless
        7. LUCENE-5399.patch
          30 kB
          Michael McCandless
        8. LUCENE-5399.patch
          28 kB
          Michael McCandless
        9. LUCENE-5399.patch
          5 kB
          Robert Muir

        Activity

          People

            Unassigned Unassigned
            rcmuir Robert Muir
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment