Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10799

Analysis slowdown with inline views and thousands of column

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Impala 3.2.0
    • Fix Version/s: Impala 4.1.0
    • Component/s: Frontend
    • Labels:
      None
    • Epic Color:
      ghx-label-10

      Description

      If there are thousands of columns in the inlineview, it‘s very slow in analysis. For example, this sql will take almost 4s in analysis if the inline view has tens of thousands of column

      select c1 from (select c1, c2... c10001 from T) T
      
      
         Query Compilation: 3s880ms
             - Translate start: 968.000ns (968.000ns)
             - Translate finished: 4.318ms (4.317ms)
             - Metadata of all 1 tables cached: 42.219ms (37.900ms)
             - Analysis finished: 3s776ms (3s734ms)
             - Value transfer graph computed: 3s806ms (30.163ms)
             - Single node plan created: 3s869ms (62.556ms)
             - Runtime filters computed: 3s874ms (5.603ms)
             - Distributed plan created: 3s874ms (128.086us)
             - Planning finished: 3s880ms (5.836ms)
      

        Attachments

          Activity

            People

            • Assignee:
              xqhe Xianqing He
              Reporter:
              xqhe Xianqing He
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: