Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10583

Result spooling hang with unbounded spooling mem limit.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 4.0.0
    • Fix Version/s: Impala 4.0.0
    • Component/s: Backend, Frontend
    • Labels:
      None
    • Epic Color:
      ghx-label-8

      Description

      Looking at BufferedPlanRootSink and SpillableRowBatchQueue, it seems the code fail to anticipate if max_spilled_result_spooling_mem = 0 (unbounded). max_unpinned_bytes_ will be set to 0 and SpillableRowBatchQueue::IsFull() will always be true. We should change max_unpinned_bytes_ to INT64_MAX if max_spilled_result_spooling_mem = 0.

      Another bug happen when setting max_result_spooling_mem = 0 (unbounded). When this happen, PlanRootSink.java will peg maxMemReservationBytes to always equal to minMemReservationBytes. I think we should override it to the default max_result_spooling_mem (100MB) or set it equal to mem_limit of the query pool.

        Attachments

          Activity

            People

            • Assignee:
              rizaon Riza Suminto
              Reporter:
              rizaon Riza Suminto
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: