Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5104

admission control memory check shouldn't fail queries with estimates equal to MEM_LIMIT

    Details

    • Epic Color:
      ghx-label-9

      Description

      I got the error

      MainThread DEBUG:concurrent_select[853]:Error running query with id b647161dd5c826da:8f641d6100000000: Rejected query from pool root.default : request memory needed 256.00 GB per node is greater than process mem limit 256.00 GB.

      which didn't make sense to me until I looked in the code and saw that the operation used was actually

      schedule->GetPerHostMemoryEstimate() >= GetProcMemLimit()

      It is a use case, at least during testing, to run a query through EXPLAIN to get the memory usage and then set MEM_LIMIT to the resulting estimate. To make this use case work, the above comparison should be changed to

      schedule->GetPerHostMemoryEstimate() > GetProcMemLimit()

      to avoid rejecting the query. The code is in admission-controller.cc

        Attachments

          Activity

            People

            • Assignee:
              bikramjeet.vig Bikramjeet Vig
              Reporter:
              mmulder Matthew Mulder
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: