Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3259

Codegen is not cancellable and can use a lot of CPU and memory

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.5.0
    • Fix Version/s: None
    • Component/s: Backend
    • Labels:

      Description

      memz stucks at

      Memory Usage
      Memory consumption / limit: 10.43 GB / 20.00 GB
      tcmalloc
      ------------------------------------------------
      MALLOC:    10489890000 (10003.9 MiB) Bytes in use by application
      MALLOC: +            0 (    0.0 MiB) Bytes in page heap freelist
      MALLOC: +    674292952 (  643.1 MiB) Bytes in central cache freelist
      MALLOC: +     15303600 (   14.6 MiB) Bytes in transfer cache freelist
      MALLOC: +     15594152 (   14.9 MiB) Bytes in thread cache freelists
      MALLOC: +     51245248 (   48.9 MiB) Bytes in malloc metadata
      MALLOC:   ------------
      MALLOC: =  11246325952 (10725.3 MiB) Actual memory used (physical + swap)
      MALLOC: +   6235471872 ( 5946.6 MiB) Bytes released to OS (aka unmapped)
      MALLOC:   ------------
      MALLOC: =  17481797824 (16671.9 MiB) Virtual address space used
      MALLOC:
      MALLOC:         565828              Spans in use
      MALLOC:            245              Thread heaps in use
      MALLOC:           8192              Tcmalloc page size
      ------------------------------------------------
      Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).
      Bytes released to the OS take up virtual address space but no physical memory.
      

      this query is cancelled because of unreachable impalad because I attached gdb..

      I have flushed the query archiving cache by issuing ~30 "show tables" queries. (this works in IMPALA-3254) but the memory in this case stayed high.

      I will try to reproduce and get a heap profile.

      There is no query running at this moment.

        Attachments

        1. screenshot-1.png
          11 kB
          Huaisi Xu

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                HuaisiXu Huaisi Xu
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: