Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5907

Increased codegen time on specific queries in Impala 2.6

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Information Provided
    • Impala 2.6.0
    • None
    • Backend
    • None

    Description

      This also repro's in latest version, but it was introduced in CDH5.8 / Impala 2.6. It happens with a large amount of OR predicates over a wide table. I've attached two profiles, the codegen part is interesting:

      2.5:

      CodeGen:(Total: 28s173ms, non-child: 28s173ms, % non-child: 100.00%)
                 - CodegenTime: 507.007ms
                 - CompileTime: 16s150ms
                 - LoadTime: 0.000ns
                 - ModuleBitcodeSize: 1.84 MB (1925128)
                 - OptimizationTime: 11s791ms
                 - PrepareTime: 202.002ms
      

      2.6:

      CodeGen:(Total: 11m36s, non-child: 11m36s, % non-child: 100.00%)
                 - CodegenTime: 40.000ms
                 - CompileTime: 15s580ms
                 - LoadTime: 0.000ns
                 - ModuleBitcodeSize: 2.19 MB (2300548)
                 - NumFunctions: 3.51K (3514)
                 - NumInstructions: 74.28K (74282)
                 - OptimizationTime: 11m20s
                 - PrepareTime: 169.000ms
      

      Attachments

        1. 571_profile.txt
          29 kB
          Balazs Jeszenszky
        2. 585_profile.txt
          29 kB
          Balazs Jeszenszky
        3. codegen_test.py
          0.8 kB
          Balazs Jeszenszky

        Issue Links

          Activity

            People

              bikramjeet.vig Bikramjeet Vig
              jeszyb Balazs Jeszenszky
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: