Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-13035

BE / Hosts Page Performance: "alerts_summary" takes too long to load

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.1
    • Fix Version/s: 2.1.2
    • Component/s: ambari-server
    • Labels:
      None

      Description

      http://c6401.ambari.apache.org:8080/api/v1/clusters/MuonBlue/hosts?fields=alerts_summary
      is taking almost 10s every time to load.

      Here is a typical request for hosts update

      http://c6401.ambari.apache.org:8080/api/v1/clusters/MuonBlue/hosts?fields=Hosts/rack_info,Hosts/host_name,
      Hosts/maintenance_state,Hosts/public_host_name,Hosts/cpu_count,Hosts/ph_cpu_count,alerts_summary,Hosts/host_status,
      Hosts/last_heartbeat_time,Hosts/ip,host_components/HostRoles/state,host_components/HostRoles/maintenance_state,
      host_components/HostRoles/stale_configs,host_components/HostRoles/service_name,host_components/HostRoles/desired_admin_state,
      Hosts/total_mem,stack_versions/HostStackVersions,stack_versions/repository_versions/RepositoryVersions/repository_version,
      stack_versions/repository_versions/RepositoryVersions/id,
      stack_versions/repository_versions/RepositoryVersions/display_name&minimal_response=true&page_size=100&from=0&sortBy=Hosts/host_name.asc&_=1439416717498
      

      By removing "alert_summary", it will take only 1/10 of the time to load.

      1. AMBARI-13035.patch
        15 kB
        Srimanth Gunturi

        Issue Links

          Activity

          Hide
          srimanth.gunturi Srimanth Gunturi added a comment -

          AlertSummaryPropertyProvider when populating 'alerts_summary' currently makes 1 sql query per host. So 1000 sql queries are made when populating this query on a 1000 node cluster, which grows linearly based on the number of hosts.

          One of the optimizations is to get alert_summaries for all hosts in 1 SQL call so that the SQL call count is constant.

          There are bigger changes that will be tracked in a different JIRA, but this JIRA will be used for optimizing the SQL call count.

          Show
          srimanth.gunturi Srimanth Gunturi added a comment - AlertSummaryPropertyProvider when populating 'alerts_summary' currently makes 1 sql query per host. So 1000 sql queries are made when populating this query on a 1000 node cluster, which grows linearly based on the number of hosts. One of the optimizations is to get alert_summaries for all hosts in 1 SQL call so that the SQL call count is constant. There are bigger changes that will be tracked in a different JIRA, but this JIRA will be used for optimizing the SQL call count.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12754703/AMBARI-13035.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in ambari-server.

          Test results: https://builds.apache.org/job/Ambari-trunk-test-patch/3739//testReport/
          Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/3739//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12754703/AMBARI-13035.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in ambari-server. Test results: https://builds.apache.org/job/Ambari-trunk-test-patch/3739//testReport/ Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/3739//console This message is automatically generated.
          Hide
          srimanth.gunturi Srimanth Gunturi added a comment -

          Committed to branch-2.1 and trunk.

          Show
          srimanth.gunturi Srimanth Gunturi added a comment - Committed to branch-2.1 and trunk.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Ambari-trunk-Commit #3411 (See https://builds.apache.org/job/Ambari-trunk-Commit/3411/)
          AMBARI-13035. BE / Hosts Page Performance: alerts_summary takes too long to load (srimanth) (sgunturi: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=b4731890272e216ab327656101ec26e051d6b9f8)

          • ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java
          • ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostAlertSummaryDTO.java
          • ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertSummaryPropertyProvider.java
          • ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Ambari-trunk-Commit #3411 (See https://builds.apache.org/job/Ambari-trunk-Commit/3411/ ) AMBARI-13035 . BE / Hosts Page Performance: alerts_summary takes too long to load (srimanth) (sgunturi: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=b4731890272e216ab327656101ec26e051d6b9f8 ) ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostAlertSummaryDTO.java ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertSummaryPropertyProvider.java ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Ambari-branch-2.1 #503 (See https://builds.apache.org/job/Ambari-branch-2.1/503/)
          AMBARI-13035. BE / Hosts Page Performance: alerts_summary takes too long to load (srimanth) (sgunturi: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=34180fb27851df7d84e682877ef2291ae452b57d)

          • ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertSummaryPropertyProvider.java
          • ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java
          • ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java
          • ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostAlertSummaryDTO.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Ambari-branch-2.1 #503 (See https://builds.apache.org/job/Ambari-branch-2.1/503/ ) AMBARI-13035 . BE / Hosts Page Performance: alerts_summary takes too long to load (srimanth) (sgunturi: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=34180fb27851df7d84e682877ef2291ae452b57d ) ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertSummaryPropertyProvider.java ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostAlertSummaryDTO.java

            People

            • Assignee:
              srimanth.gunturi Srimanth Gunturi
              Reporter:
              srimanth.gunturi Srimanth Gunturi
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development