Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1616

Improve the Memory Limit Exceeded error report

    XMLWordPrintableJSON

    Details

      Description

      From a question in the user list:

      As part of debugging that I'm trying to get better at reading the profile. One thing that's still unclear to me is that I see a few fragments with Peak Mem in the 800MB neighborhood (on machines with 13GB memory limit :-\ ) but I can't really tell which fragment caused failure. Is there somewhere in the profile I can determine the failed fragment?

      I see "Limit: Consumption=1.41GB". Does this mean there is no limit?

      I only see one big fragment (604MB) in this, but profile ExecSummary shows two fragments over 800MB.

      Backend 218:Memory Limit Exceeded
      Query(5e4656620071100e:b75fbeeac47e3d83) Limit: Consumption=1.41 GB
        Fragment 5e4656620071100e:b75fbeeac47e3d85: Consumption=12.53 MB
          UDFs: Consumption=0
          HASH_JOIN_NODE (id=16): Consumption=8.00 KB
          AGGREGATION_NODE (id=7): Consumption=6.25 MB
          AGGREGATION_NODE (id=24): Consumption=6.25 MB
          EXCHANGE_NODE (id=23): Consumption=0
          DataStreamRecvr: Consumption=0
          EXCHANGE_NODE (id=33): Consumption=0
          DataStreamRecvr: Consumption=0
          DataStreamSender: Consumption=4.00 KB
        Block Manager: Limit=10.40 GB Consumption=1.31 GB
        Fragment 5e4656620071100e:b75fbeeac47e3dba: Consumption=12.71 MB
          UDFs: Consumption=0
          AGGREGATION_NODE (id=15): Consumption=6.25 MB
          AGGREGATION_NODE (id=32): Consumption=6.25 MB
          EXCHANGE_NODE (id=31): Consumption=0
          DataStreamRecvr: Consumption=0
          DataStreamSender: Consumption=212.00 KB
        Fragment 5e4656620071100e:b75fbeeac47e3def: Consumption=10.49 MB
          UDFs: Consumption=8.00 KB
          AGGREGATION_NODE (id=14): Consumption=6.25 MB
          HASH_JOIN_NODE (id=13): Consumption=4.02 MB
          EXCHANGE_NODE (id=29): Consumption=0
          DataStreamRecvr: Consumption=0
          EXCHANGE_NODE (id=30): Consumption=0
          DataStreamRecvr: Consumption=0
          DataStreamSender: Consumption=212.00 KB
        Fragment 5e4656620071100e:b75fbeeac47e3e28: Consumption=24.11 MB
          UDFs: Consumption=0
          HASH_JOIN_NODE (id=11): Consumption=20.04 MB
          EXCHANGE_NODE (id=27): Consumption=0
          DataStreamRecvr: Consumption=3.85 MB
          EXCHANGE_NODE (id=28): Consumption=0
          DataStreamRecvr: Consumption=5.33 KB
          DataStreamSender: Consumption=212.00 KB
        Fragment 5e4656620071100e:b75fbeeac47e3e5f: Consumption=1.11 MB
          UDFs: Consumption=0
          AGGREGATION_NODE (id=26): Consumption=0
          EXCHANGE_NODE (id=25): Consumption=0
          DataStreamSender: Consumption=1.10 MB
        Fragment 5e4656620071100e:b75fbeeac47e3e9c: Consumption=10.56 MB
          UDFs: Consumption=4.00 KB
          AGGREGATION_NODE (id=6): Consumption=6.25 MB
          HASH_JOIN_NODE (id=5): Consumption=4.02 MB
          EXCHANGE_NODE (id=21): Consumption=0
          DataStreamRecvr: Consumption=0
          EXCHANGE_NODE (id=22): Consumption=0
          DataStreamRecvr: Consumption=0
          DataStreamSender: Consumption=282.39 KB
        Fragment 5e4656620071100e:b75fbeeac47e3ed5: Consumption=1.72 MB
          UDFs: Consumption=0
          HASH_JOIN_NODE (id=3): Consumption=8.00 KB
          EXCHANGE_NODE (id=19): Consumption=0
          DataStreamRecvr: Consumption=1.43 MB
          EXCHANGE_NODE (id=20): Consumption=0
          DataStreamRecvr: Consumption=0
          DataStreamSender: Consumption=282.39 KB
        Fragment 5e4656620071100e:b75fbeeac47e3f0c: Consumption=604.83 MB
          UDFs: Consumption=0
          AGGREGATION_NODE (id=18): Consumption=604.40 MB
          EXCHANGE_NODE (id=17): Consumption=0
          DataStreamRecvr: Consumption=141.46 KB
          DataStreamSender: Consumption=282.39 KB
        Fragment 5e4656620071100e:b75fbeeac47e3f14: Consumption=460.89 KB
          UDFs: Consumption=0
          HDFS_SCAN_NODE (id=0): Consumption=0
          DataStreamSender: Consumption=452.89 KB
      

        Attachments

          Activity

            People

            • Assignee:
              attilaj Attila Jeges
              Reporter:
              ippokratis Ippokratis Pandis
            • Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: