Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1552

Improve info shown in the OOM message

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Duplicate
    • Affects Version/s: Impala 2.0
    • Fix Version/s: None
    • Component/s: None

      Description

      When running out of memory in PAGG the error message does not make it
      clear why we ran out of memory. In the example below we see that the
      limit is 100MB and the consumption is only 8MB, but we ran out of memory.
      In this particular case, the reason is that PAGG tries to allocate the
      initial min mem for each partition and fails (each partition needs ~8MB
      and we have 32 of them).

      It would be good to improve the error message with some more useful info,
      e.g. how much more memory was needed.

      Backend 0:Memory Limit Exceeded
      Query(3b428a0f1e22f611:722a79f624fcb0b5) Limit: Limit=100.00 MB Consumption=8.03 MB
      Fragment 3b428a0f1e22f611:722a79f624fcb0b6: Consumption=8.00 KB
      EXCHANGE_NODE (id=5): Consumption=0
      DataStreamRecvr: Consumption=0
      Block Manager: Consumption=0
      Fragment 3b428a0f1e22f611:722a79f624fcb0b7: Consumption=4.01 MB
      SORT_NODE (id=2): Consumption=0
      AGGREGATION_NODE (id=4): Consumption=4.00 MB
      EXCHANGE_NODE (id=3): Consumption=0
      DataStreamRecvr: Consumption=0
      DataStreamSender: Consumption=4.00 KB
      Fragment 3b428a0f1e22f611:722a79f624fcb0b8: Consumption=4.01 MB
      AGGREGATION_NODE (id=1): Consumption=4.00 MB
      HDFS_SCAN_NODE (id=0): Consumption=0
      DataStreamSender: Consumption=4.00 KB

        Attachments

          Activity

            People

            • Assignee:
              HuaisiXu Huaisi Xu
              Reporter:
              ippokratis Ippokratis Pandis
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: