[AMBARI-10776] blueprint topology validation doesn't properly handle the 'auto deploy' component dependency graph

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.0
• Component/s: blueprints
• Assignee: John Speidel
• Reporter: John Speidel
• Created: 27/Apr/15 20:26
• Updated: 27/Mar/17 18:43
• Resolved: 29/Apr/15 03:20

Description

When a blueprint is submitted, topology validation is performed. During this validation, the blueprint validator ensures that the blueprint topology is valid based on the component cardinalities specified in the stack.

Some components are specified as being "auto-deploy" enabled in the stack. This means that if the component in not explicitly specified in the blueprint that it will be added according to the information provided in the stack. In the case where a component is added because it wasn't explicitly specified and it is auto-deploy enabled, if it has dependencies itself, these should be validated and this isn't occurring.

For example, in a BP that specified HIVE_SERVER, but not HIVE_METASTORE, the metastore component is automatically added to the blueprint because it is auto-deploy enabled. HIVE_METASTORE itself has a dependency on MYSQL_SERVER and because it is also auto-deploy enabled, it should also be added to the blueprint but currently validation doesn't occur after adding HIVE_METASTORE so it's dependencies are not validated and no auto-deploy occurs.

This is a regression that occurred in the patch for AMBARI-10750.

Example blueprint which demonstrates this issue:

{    
  "host_groups" : [
    {
      "name" : "master",      
      "components" : [       
        {
          "name" : "HISTORYSERVER"
        },        
        {
          "name" : "AMBARI_SERVER"
        },
        {
          "name" : "APP_TIMELINE_SERVER"
        },
        {
          "name" : "RESOURCEMANAGER"
        },        
        {
          "name" : "ZOOKEEPER_CLIENT"
        },
        {
          "name" : "MAPREDUCE2_CLIENT"
        },
        {
          "name" : "YARN_CLIENT"
        },
        {
          "name" : "HIVE_SERVER"
        },
        {
          "name" : "HIVE_CLIENT"
        },
        {
          "name" : "WEBHCAT_SERVER"
        }
      ],
      "cardinality" : "1"
    },
    {
      "name" : "master2",      
      "components" : [  
        {
          "name" : "NAMENODE"
        },     
        {
          "name": "ZOOKEEPER_SERVER"
        },
        {
          "name" : "SECONDARY_NAMENODE"
        },  
        {
          "name" : "ZOOKEEPER_CLIENT"
        },
        {
          "name" : "MAPREDUCE2_CLIENT"
        },
        {
          "name" : "YARN_CLIENT"
        }
      ]
    },    
    {      
      "name" : "slave",      
      "components" : [
        {
          "name" : "NODEMANAGER"
        },
        {
          "name" : "DATANODE"
        },   
        {
          "name" : "YARN_CLIENT"
        },
        {
          "name" : "ZOOKEEPER_CLIENT"
        },
        {
          "name" : "MAPREDUCE2_CLIENT"
        }
      ]
    }
  ],
  "Blueprints" : {
    "stack_name" : "HDP",
    "stack_version" : "2.2"
  }
}

After the blueprint is created you will see that HIVE_METASTORE has been added but not MYSQL_SERVER which causes a failure during topology configuration update.

Issue Links

• is related to: AMBARI-20594 Validate hive service dependencies based on the configuration on blueprint deployments