Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5013

Re-evaluate our approach to per-operator memory estimates

    XMLWordPrintableJSON

Details

    Description

      The way that memory estimates are computed for PlanNodes and Sinks are ad-hoc and in some cases much less accurate than they could be. We should clarify what the memory estimates mean, how they should be computed and then systematically fix them.

      In general it's difficult to produce accurate memory estimates, because it depends on having accurate estimates of cardinality and other runtime parameters, so this JIRA isn't meant to guarantee any specific level of accuracy of estimates, just to generally improve the estimates and clarify what they mean and how they should be calculated

      We should also consider deprecating or removing these estimates, unless they are useful for computing "ideal" memory in IMPALA-3706.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: