Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4291

Lazy creation of IRFunction::Type to Function* mapping

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Backend
    • Labels:
    • Docs Text:
      This change reduces the preparation time during codegen of a simple query from 23ms to 10ms. Reduces the latency of short-running query.

      Description

      For simple queries with one or two functions to codegen, we spent about 23ms or more out of 40ms of codegen time in preparation. We can save some time by not eagerly populating the map of IRFunction::Type to llvm::Function* map. This should help with the fixes for IMPALA-3638 when we create the LLVM module unconditionally.

        Activity

        Hide
        kwho Michael Ho added a comment -

        https://github.com/apache/incubator-impala/commit/47b8aa3a9e7682ebb182696901916900d3323039

        IMPALA-4291: Reduce LLVM module's preparation time
        Previously, when creating a LlvmCodeGen object, we
        run an O(mn) algorithm to map the IRFunction::Type
        to the actual LLVM::Function object in the module.
        m is the size of IRFunction::Type enum and n is
        the total number of functions in the module. This
        is a waste of time if we only use few functions
        from the module.

        This change reduces the preparation time of a simple
        query from 23ms to 10ms.

        select count from tpch100_parquet.lineitem where l_orderkey > 20;

        Show
        kwho Michael Ho added a comment - https://github.com/apache/incubator-impala/commit/47b8aa3a9e7682ebb182696901916900d3323039 IMPALA-4291 : Reduce LLVM module's preparation time Previously, when creating a LlvmCodeGen object, we run an O(mn) algorithm to map the IRFunction::Type to the actual LLVM::Function object in the module. m is the size of IRFunction::Type enum and n is the total number of functions in the module. This is a waste of time if we only use few functions from the module. This change reduces the preparation time of a simple query from 23ms to 10ms. select count from tpch100_parquet.lineitem where l_orderkey > 20;

          People

          • Assignee:
            kwho Michael Ho
            Reporter:
            kwho Michael Ho
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development