Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10036

Admission control incorrectly rejecting query based on coordinator limit

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 4.0.0
    • Impala 4.0.0
    • Backend
    • None
    • ghx-label-5

    Description

      Admission control rejected a query with the following error:

      ERROR: Rejected query from pool root.default: request memory needed 50.00 GB is greater than memory available for admission 45.36 GB of coordinator-0:22050.
      

      The query profile shows the following resource estimates:

      Per-Host Resource Estimates: Memory=303.36GB
      Dedicated Coordinator Resource Estimate: Memory=136MB
      

      The max query memory limit for the pool was configured as 50 GB.

      Based on the coordinator resource estimate of 136 MB the query should have easily fit in the available coordinator memory. tarmstrong speculated that we might simply be comparing the wrong resource estimate to the coordinator limit.

      Profile attached.

      Attachments

        1. 23_profile.txt
          57 kB
          David Rorke
        2. backends-repro.png
          115 kB
          Tim Armstrong
        3. impalad.INFO.gz
          226 kB
          Tim Armstrong
        4. start-ded-coord-cluster.sh
          0.4 kB
          Tim Armstrong
        5. tmp-fair-scheduler.xml
          2 kB
          Tim Armstrong
        6. tmp-llama-site.xml
          3 kB
          Tim Armstrong

        Activity

          People

            tarmstrong Tim Armstrong
            drorke David Rorke
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: