Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5104

admission control memory check shouldn't fail queries with estimates equal to MEM_LIMIT

    XMLWordPrintableJSON

Details

    • ghx-label-9

    Description

      I got the error

      MainThread DEBUG:concurrent_select[853]:Error running query with id b647161dd5c826da:8f641d6100000000: Rejected query from pool root.default : request memory needed 256.00 GB per node is greater than process mem limit 256.00 GB.

      which didn't make sense to me until I looked in the code and saw that the operation used was actually

      schedule->GetPerHostMemoryEstimate() >= GetProcMemLimit()

      It is a use case, at least during testing, to run a query through EXPLAIN to get the memory usage and then set MEM_LIMIT to the resulting estimate. To make this use case work, the above comparison should be changed to

      schedule->GetPerHostMemoryEstimate() > GetProcMemLimit()

      to avoid rejecting the query. The code is in admission-controller.cc

      Attachments

        Activity

          People

            bikramjeet.vig Bikramjeet Vig
            mmulder Matthew Mulder
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: