Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7096

Ensure no memory limit exceeded regressions from IMPALA-4835 because of non-reserved memory

    XMLWordPrintableJSON

    Details

      Description

      IMPALA-7078 showed some cases where non-buffer memory could accumulate in the row batch queue and cause memory consumption problems.

      The decision for whether to spin up a scanner thread in IMPALA-4835 implicitly assumes that buffer memory is the bulk of memory consumed by a scan, but there may be cases where that is not true and the previous heuristic would be more conservative about starting a scanner thread.

      We should investigate this further and figure out how to avoid it if there's an issue.

        Attachments

        1. ScanConsumingMostMemory.txt
          37 kB
          Mostafa Mokhtar

          Issue Links

            Activity

              People

              • Assignee:
                tarmstrong Tim Armstrong
                Reporter:
                tarmstrong Tim Armstrong
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: