Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-9002

query caching leads to absurdly slow queries

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 7.7.2, 8.2
    • Fix Version/s: None
    • Component/s: core/search
    • Labels:
    • Lucene Fields:
      New

      Description

      Description
      We have dozens of ES clusters(based on Lucene) for metric scenarios. Most of the queries are like this: host_ip:10.10.10.10 AND timestamp:[2019-10-01 00:00:00 TO 2019-10-05 23:59:59]. And we frequently encounter some absurdly slow queries.

      Solution
      For a long time range query(e.g. 5 days), each range query will consume tens of megabytes of memory and spend hundreds of milliseconds to cache, but the benefits are not obvious. And those large cache entries will cause frequent cache eviction. So it's better to skip the caching action directly when large range query appears with a selective lead iterator.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jgq2008303393 Guoqiang Jiang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 9h 20m
                  9h 20m