Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-5105

SCM is OOM due to too many ContainerReport is queued.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.2.0
    • Fix Version/s: None
    • Component/s: SCM
    • Labels:
      None

      Description

      The heap dump 

      https://drive.google.com/file/d/1k4rRSDv6lazsViwTouF8MXSPLF_AvY6w/view?usp=sharing

       

      Some Info,
      1. SCM's  max heap is 10GB. SCM is OOM after it's first restart after a cluster wise upgrade and restart.  Recon is on at that time.  Then we restart the SCM and turn off the Recon. SCM works well after that.
      2. There are totally 400K+ containers in the cluster of 40 DNs.
       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              glengeng Glen Geng
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: