Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1910

Significant memory leak in HashJoin Node

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.1.1
    • Fix Version/s: Impala 2.2
    • Component/s: None
    • Labels:
      None

      Description

      Here's the leak:

      Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
      F0320 20:41:54.770300 28819 exec-node.cc:175] Check failed: mem_tracker()->consumption() == 0 (1539309568 vs. 0) Leaked memory.
      Fragment af4cf08b2927df82:dc517e368ba78aa4: Consumption=1.43 GB
      AGGREGATION_NODE (id=3): Consumption=0
      HASH_JOIN_NODE (id=2): Consumption=1.43 GB
      EXCHANGE_NODE (id=4): Consumption=0
      EXCHANGE_NODE (id=5): Consumption=0
      DataStreamSender: Consumption=16.00 KB

      The leak is a lot more than what's reported in IMPALA-1867.

      The query that caused the leak is in this form:

      select * from small_tbl A, big_tbl B where A.id=B.id

      The per query mem_limit is set to ~1.5gb.

      We can use tpc-ds to try to repro it.

        Attachments

        1. impalad (9).INFO
          126 kB
          Alan Choi

          Issue Links

            Activity

              People

              • Assignee:
                dhecht Dan Hecht
                Reporter:
                alan@cloudera.com Alan Choi
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: