Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7367

Pack StringValue, CollectionValue and TimestampValue slots

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Impala 3.2.0
    • Component/s: Backend
    • Labels:

      Description

      This is a follow-on to finish up the work from IMPALA-2789. IMPALA-2789 didn't actually fully pack the memory layout because StringValue, TimestampValue and CollectionValue still occupy 16 bytes but only have 12 bytes of actual data. This results in a higher memory footprint, which leads to higher memory requirements and worse performance. We don't get any benefit from the padding since the majority of tuples are not actually aligned in memory anyway.

      I did a quick version of the change for StringValue only which improves TPC-H performance.

      Report Generated on 2018-07-30
      Run Description: "b5608264b4552e44eb73ded1e232a8775c3dba6b vs f1e401505ac20c0400eec819b9196f7f506fb927"
      
      Cluster Name: UNKNOWN
      Lab Run Info: UNKNOWN
      Impala Version:          impalad version 3.1.0-SNAPSHOT RELEASE ()
      Baseline Impala Version: impalad version 3.1.0-SNAPSHOT RELEASE (2018-07-27)
      
      +----------+-----------------------+---------+------------+------------+----------------+
      | Workload | File Format           | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
      +----------+-----------------------+---------+------------+------------+----------------+
      | TPCH(10) | parquet / none / none | 2.69    | -4.78%     | 2.09       | -3.11%         |
      +----------+-----------------------+---------+------------+------------+----------------+
      
      +----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
      | Workload | Query    | File Format           | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%)  | Base StdDev(%) | Num Clients | Iters |
      +----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
      | TPCH(10) | TPCH-Q22 | parquet / none / none | 0.94   | 0.93        |   +0.75%   |   3.37%    |   2.84%        | 1           | 30    |
      | TPCH(10) | TPCH-Q13 | parquet / none / none | 3.32   | 3.32        |   +0.13%   |   1.74%    |   2.09%        | 1           | 30    |
      | TPCH(10) | TPCH-Q11 | parquet / none / none | 0.99   | 0.99        |   -0.02%   |   3.74%    |   3.16%        | 1           | 30    |
      | TPCH(10) | TPCH-Q5  | parquet / none / none | 2.30   | 2.33        |   -0.96%   |   2.15%    |   2.45%        | 1           | 30    |
      | TPCH(10) | TPCH-Q2  | parquet / none / none | 1.55   | 1.57        |   -1.45%   |   1.65%    |   1.49%        | 1           | 30    |
      | TPCH(10) | TPCH-Q8  | parquet / none / none | 2.89   | 2.93        |   -1.51%   |   2.69%    |   1.34%        | 1           | 30    |
      | TPCH(10) | TPCH-Q9  | parquet / none / none | 5.96   | 6.06        |   -1.63%   |   1.34%    |   1.82%        | 1           | 30    |
      | TPCH(10) | TPCH-Q20 | parquet / none / none | 1.58   | 1.61        |   -1.85%   |   2.28%    |   2.16%        | 1           | 30    |
      | TPCH(10) | TPCH-Q16 | parquet / none / none | 1.18   | 1.21        |   -2.11%   |   3.68%    |   4.72%        | 1           | 30    |
      | TPCH(10) | TPCH-Q3  | parquet / none / none | 2.13   | 2.18        |   -2.31%   |   2.09%    |   1.92%        | 1           | 30    |
      | TPCH(10) | TPCH-Q15 | parquet / none / none | 1.86   | 1.90        |   -2.52%   |   2.06%    |   2.22%        | 1           | 30    |
      | TPCH(10) | TPCH-Q17 | parquet / none / none | 1.85   | 1.90        |   -2.86%   |   10.00%   |   8.02%        | 1           | 30    |
      | TPCH(10) | TPCH-Q10 | parquet / none / none | 2.58   | 2.66        |   -2.93%   |   1.68%    |   6.49%        | 1           | 30    |
      | TPCH(10) | TPCH-Q14 | parquet / none / none | 1.37   | 1.42        |   -3.22%   |   3.35%    |   6.24%        | 1           | 30    |
      | TPCH(10) | TPCH-Q18 | parquet / none / none | 4.99   | 5.17        |   -3.38%   |   1.75%    |   3.82%        | 1           | 30    |
      | TPCH(10) | TPCH-Q6  | parquet / none / none | 0.66   | 0.69        |   -3.73%   |   5.04%    |   4.12%        | 1           | 30    |
      | TPCH(10) | TPCH-Q4  | parquet / none / none | 1.07   | 1.12        |   -3.97%   |   1.79%    |   2.85%        | 1           | 30    |
      | TPCH(10) | TPCH-Q1  | parquet / none / none | 2.29   | 2.39        |   -4.34%   |   2.70%    |   3.43%        | 1           | 30    |
      | TPCH(10) | TPCH-Q7  | parquet / none / none | 6.17   | 6.45        |   -4.42%   |   2.15%    |   1.69%        | 1           | 30    |
      | TPCH(10) | TPCH-Q19 | parquet / none / none | 1.85   | 1.93        |   -4.42%   |   2.76%    |   2.26%        | 1           | 30    |
      | TPCH(10) | TPCH-Q12 | parquet / none / none | 1.32   | 1.42        |   -6.75%   |   2.55%    |   3.56%        | 1           | 30    |
      | TPCH(10) | TPCH-Q21 | parquet / none / none | 10.43  | 12.08       |   -13.69%  | * 14.21% * | * 10.76% *     | 1           | 30    |
      +----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
      

        Attachments

        1. 0001-WIP.patch
          7 kB
          Tim Armstrong

          Issue Links

            Activity

              People

              • Assignee:
                poojanilangekar Pooja Nilangekar
                Reporter:
                tarmstrong Tim Armstrong
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: