Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3700

Introduce upper limit to hash table size or HashBuckets for Streaming agg

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: Impala 2.7.0
    • Fix Version/s: None
    • Component/s: Backend

      Description

      Several queries consume all the memory limit in Streaming agg leaving very little memory left for other operators.

      Attached is the profile for TPC-DS Q67 which fails for the reason mentioned.

      An alternative solution is to extend the operator interface to respond to memory pressure by spilling and releasing some memory back to the memory/block manager.

        Attachments

        1. query67.sql.1.profile.out
          5.33 MB
          Mostafa Mokhtar
        2. query67.sql.1.out
          10 kB
          Mostafa Mokhtar

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mmokhtar Mostafa Mokhtar
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: