Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-20875

Removing A Service Causes DB Verification To Produce Warnings

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.2
    • Fix Version/s: 2.5.2
    • Component/s: ambari-server
    • Labels:
      None

      Description

      When removing a service, the configurations for that service are kept for historical purposes, but their various associations in the database are removed (specifically, the serviceconfigmapping relationships).

      After removing a service, the orphaned configurations now cause a warning to be displayed on Ambari Server startup:

      2017-04-06 17:15:24,003  WARN - You have config(s): ranger-storm-policymgr-ssl-version1467149286586,atlas-env-version1471883877194,falcon-env-version1467044148480,storm-site-version1467149286586,storm-site-version1474944944095,ranger-storm-plugin-properties-version1467149286586,hana_hadoop-env-version1476989318735,hana_hadoop-env-version1468951412523,hanaes-site-version1475773173499,hanaes-site-version1477639131416,atlas-env-version1471880496396,falcon-startup.properties-version1474944962583,ranger-storm-security-version1467149286586,falcon-env-version1474944962517,application-properties-version1471883877194,hanaes-site-version1468951412523,application-properties-version1471992143777,application-properties-version1471880496396,hana_hadoop-env-version1475790068354,hana_hadoop-env-version1477639131416,falcon-runtime.properties-version1467044148480,atlas-env-version1471992143777,hana_hadoop-env-version1475773173499,storm-env-version1467149286586,hanaes-site-version1475790068354,hanaes-site-version1476902714170,atlas-env-version1471883827584,hana_hadoop-env-version1477695406433,hanaes-site-version1476989583427,falcon-log4j-version1,falcon-env-version1474944962457,hanaes-site-version1468959251565,falcon-client.properties-version1,atlas-env-version1471993347065,falcon-startup.properties-version1467044148480,storm-cluster-log4j-version1467149286586,hanaes-site-version1472285532383,hana_hadoop-env-version1477695089738,hana_hadoop-env-version1468959251565,hana_hadoop-env-version1476989821279,atlas-log4j-version1,storm-site-version1467612840864,storm-worker-log4j-version1467149286586,ranger-storm-audit-version1467149286586,application-properties-version1471993347065,application-properties-version1471883827584,hana_hadoop-env-version1477695579450 that is(are) not mapped (in serviceconfigmapping table) to any service!
      

      These orphaned configurations have entries in both clusterconfig and clusterconfigmapping but are otherwise not references anywhere. They don't hurt anything, but do trigger this warning since we can't determine if they should have mappings in serviceconfigmapping.

      A few options:

      • When removing a service, remove configurations as well, leaving no orphans. Some would argue that this is a bad move since re-adding the service later would allow you to see the old configurations. I do not believe this is true since the old configurations are never associated with the new service's serviceconfig or serviceconfigmapping.
      • Make the warning smarter somehow to ignore these since it's expected they are orphaned.
        • Somehow determine the service which should own the config and see if it exists in the cluster?
        • Add a new column to clusterconfig to mark it as deleted?

      To clean these warnings, we had to:

      CREATE TEMPORARY TABLE IF NOT EXISTS orphaned_configs AS
      (SELECT
      cc.config_id,
      cc.type_name,
      cc.version_tag
      FROM clusterconfig cc, clusterconfigmapping ccm
      WHERE cc.config_id NOT IN (SELECT
      scm.config_id
      FROM serviceconfigmapping scm)
      AND cc.type_name != 'cluster-env'
      AND cc.type_name = ccm.type_name
      AND cc.version_tag = ccm.version_tag);
      
      DELETE FROM clusterconfigmapping
      WHERE EXISTS
      (SELECT 1 FROM orphaned_configs
      WHERE clusterconfigmapping.type_name = orphaned_configs.type_name AND clusterconfigmapping.version_tag = orphaned_configs.version_tag);
      
      DELETE FROM clusterconfig WHERE clusterconfig.config_id IN (SELECT config_id FROM orphaned_configs);
      
      SELECT * FROM orphaned_configs;
      
      DROP TABLE orphaned_configs;
      

      I've considered advanced heuristics based on service metainfo with config dependencies, and service config mappings. But this approach may be unreliable when given inaccurate service metainfo between stack upgrades. Also, this approach is likely to delay server start on clusters with thousands of configs.
      So the solution Add a new column to clusterconfig to mark it as deleted. So warning will not be generated for such cluster configs.

        Attachments

        1. AMBARI-20875.patch
          20 kB
          Dmitry Lysnichenko

          Issue Links

            Activity

              People

              • Assignee:
                dmitriusan Dmitry Lysnichenko
                Reporter:
                dmitriusan Dmitry Lysnichenko
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: