Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-13307

Agent instance memory footprint likely gradually growing

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2.0
    • Component/s: None
    • Labels:
      None

      Description

      Noticed about 1.5 G of resident memory for the agents on a cluster that is running for about a month.
      Which is an obvious memory leak.

        Issue Links

          Activity

          Hide
          aonishuk Andrew Onischuk added a comment -

          Committed to trunk and branch-2.1

          Show
          aonishuk Andrew Onischuk added a comment - Committed to trunk and branch-2.1
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Ambari-branch-2.1 #635 (See https://builds.apache.org/job/Ambari-branch-2.1/635/)
          AMBARI-13307. Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=13f808e3cb7bd27af2781757e97bfd8ea540308c)

          • ambari-common/src/main/python/resource_management/core/logger.py
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Ambari-branch-2.1 #635 (See https://builds.apache.org/job/Ambari-branch-2.1/635/ ) AMBARI-13307 . Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=13f808e3cb7bd27af2781757e97bfd8ea540308c ) ambari-common/src/main/python/resource_management/core/logger.py
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Ambari-trunk-Commit #3594 (See https://builds.apache.org/job/Ambari-trunk-Commit/3594/)
          AMBARI-13307. Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=b8be80b08cf50436fb1285b5deaa6a4ca8fe8739)

          • ambari-common/src/main/python/resource_management/core/logger.py
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Ambari-trunk-Commit #3594 (See https://builds.apache.org/job/Ambari-trunk-Commit/3594/ ) AMBARI-13307 . Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=b8be80b08cf50436fb1285b5deaa6a4ca8fe8739 ) ambari-common/src/main/python/resource_management/core/logger.py
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Ambari-branch-2.1 #644 (See https://builds.apache.org/job/Ambari-branch-2.1/644/)
          AMBARI-13307. Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=32a5c51c90b0c522be2ea640d34e38b11849550a)

          • ambari-common/src/main/python/resource_management/core/logger.py
          • ambari-common/src/main/python/resource_management/libraries/script/script.py
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Ambari-branch-2.1 #644 (See https://builds.apache.org/job/Ambari-branch-2.1/644/ ) AMBARI-13307 . Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=32a5c51c90b0c522be2ea640d34e38b11849550a ) ambari-common/src/main/python/resource_management/core/logger.py ambari-common/src/main/python/resource_management/libraries/script/script.py
          Hide
          jonathan.hurley Jonathan Hurley added a comment -

          I had to revert this commit. It was preventing status commands from running which would show all components as stopped even though they were running and alerts were green.

          I even deployed a new cluster thinking that it was my environment. Verified that reverting this patch resolves the problem

          When I ran the status command manually on the command line, I noticed zero output from the logger, so I figured that was a good place to start. Noticed this commit which seemed to be the cause.

          Show
          jonathan.hurley Jonathan Hurley added a comment - I had to revert this commit. It was preventing status commands from running which would show all components as stopped even though they were running and alerts were green. I even deployed a new cluster thinking that it was my environment. Verified that reverting this patch resolves the problem When I ran the status command manually on the command line, I noticed zero output from the logger, so I figured that was a good place to start. Noticed this commit which seemed to be the cause.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Ambari-branch-2.1 #650 (See https://builds.apache.org/job/Ambari-branch-2.1/650/)
          Revert "AMBARI-13307. Agent instance memory footprint likely gradually (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=6930aafa22342d650b94c0c12203daa7f30b819f)

          Show
          hudson Hudson added a comment - FAILURE: Integrated in Ambari-branch-2.1 #650 (See https://builds.apache.org/job/Ambari-branch-2.1/650/ ) Revert " AMBARI-13307 . Agent instance memory footprint likely gradually (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=6930aafa22342d650b94c0c12203daa7f30b819f ) ambari-common/src/main/python/resource_management/libraries/script/script.py ambari-common/src/main/python/resource_management/core/logger.py Revert " AMBARI-13307 . Agent instance memory footprint likely gradually (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=68395ded0d2f6174b9f13a6b6ad0c800308b7d74 ) ambari-common/src/main/python/resource_management/core/logger.py
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Ambari-trunk-Commit #3610 (See https://builds.apache.org/job/Ambari-trunk-Commit/3610/)
          Revert "AMBARI-13307. Agent instance memory footprint likely gradually (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=3c8bf9de2afaab2e4894e875d9f2cd44a9a7eab6)

          • ambari-common/src/main/python/resource_management/core/logger.py
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Ambari-trunk-Commit #3610 (See https://builds.apache.org/job/Ambari-trunk-Commit/3610/ ) Revert " AMBARI-13307 . Agent instance memory footprint likely gradually (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=3c8bf9de2afaab2e4894e875d9f2cd44a9a7eab6 ) ambari-common/src/main/python/resource_management/core/logger.py
          Hide
          aonishuk Andrew Onischuk added a comment -

          Committed to trunk

          Show
          aonishuk Andrew Onischuk added a comment - Committed to trunk
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Ambari-branch-2.1 #653 (See https://builds.apache.org/job/Ambari-branch-2.1/653/)
          Revert "Revert "AMBARI-13307. Agent instance memory footprint likely (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=dee7089b1cbec527c56bdd8925fae9e421ef7b10)

          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Ambari-branch-2.1 #653 (See https://builds.apache.org/job/Ambari-branch-2.1/653/ ) Revert "Revert " AMBARI-13307 . Agent instance memory footprint likely (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=dee7089b1cbec527c56bdd8925fae9e421ef7b10 ) ambari-common/src/main/python/resource_management/core/logger.py Revert "Revert " AMBARI-13307 . Agent instance memory footprint likely (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=8ef722dc3f60e1b6100be49219df6799203cf49c ) ambari-common/src/main/python/resource_management/libraries/script/script.py ambari-common/src/main/python/resource_management/core/logger.py
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Ambari-trunk-Commit #3613 (See https://builds.apache.org/job/Ambari-trunk-Commit/3613/)
          AMBARI-13307. Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=c6303b67b363790128eea12f99e7d8ad14ecb6d7)

          • ambari-common/src/main/python/resource_management/libraries/script/script.py
          • ambari-common/src/main/python/resource_management/core/logger.py
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Ambari-trunk-Commit #3613 (See https://builds.apache.org/job/Ambari-trunk-Commit/3613/ ) AMBARI-13307 . Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=c6303b67b363790128eea12f99e7d8ad14ecb6d7 ) ambari-common/src/main/python/resource_management/libraries/script/script.py ambari-common/src/main/python/resource_management/core/logger.py
          Hide
          aonishuk Andrew Onischuk added a comment -

          Seems like the patch was breaking tests on BAO.
          Added fix for this. Please wait for the next build.

          Show
          aonishuk Andrew Onischuk added a comment - Seems like the patch was breaking tests on BAO. Added fix for this. Please wait for the next build.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Ambari-trunk-Commit #3616 (See https://builds.apache.org/job/Ambari-trunk-Commit/3616/)
          AMBARI-13307. Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=664bde1c6e15961b39eb673b337be9f7bc6a50d2)

          • ambari-common/src/main/python/resource_management/libraries/script/script.py
          • ambari-server/src/test/python/stacks/2.0.6/FLUME/test_flume.py
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Ambari-trunk-Commit #3616 (See https://builds.apache.org/job/Ambari-trunk-Commit/3616/ ) AMBARI-13307 . Agent instance memory footprint likely gradually growing (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=664bde1c6e15961b39eb673b337be9f7bc6a50d2 ) ambari-common/src/main/python/resource_management/libraries/script/script.py ambari-server/src/test/python/stacks/2.0.6/FLUME/test_flume.py

            People

            • Assignee:
              aonishuk Andrew Onischuk
              Reporter:
              aonishuk Andrew Onischuk
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development