Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1150

Impala should limit the per-query core estimate to Yarn's per-node limit

    Details

      Description

      Yarn has yarn.nodemanager.resource.cpu-vcores which sets the maximum number of cores a single node can allocate. Impala uses Java's availableCores() method which looks at the whole machine. When RM is enabled, we should limit the estimated number of cores per-node to yarn.nodemanager.resource.cpu-vcores.

      This is probably easiest to do in the backend in QuerySchedule::GetPerHostVCores().

        Attachments

          Activity

            People

            • Assignee:
              mjacobs Matthew Jacobs
              Reporter:
              henryr Henry Robinson
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: