Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3797

Too many or filters may break Kylin server when flatting filter

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v3.0.0-alpha, v2.6.4
    • Component/s: Query Engine
    • Labels:
      None

      Description

      Kylin needs to convert filter into a flat format, like: OR(AND(f1,f2), AND(f3,f4,f5)…) , so that Kylin can build HBase scan/filter accordingly, but when there are too many or filters in query, like: AND(OR(f1,f2,...,f1000), OR(g1,g2,...,g1000), OR(h1,h2,...,h1000)), then the generated flat filter size will be 1000*1000*1000=1 billion, the flatting process will cause Kylin server OOM.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                magang Gang Ma
                Reporter:
                magang Gang Ma
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: