[KNOX-2160] Introduce a Hadoop XML type descriptor format

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0, 1.2.0, 1.3.0
• Fix Version/s: 1.4.0
• Component/s: Server
• Assignee: Sandor Molnar
• Reporter: Sandor Molnar
• Created: 08/Jan/20 08:30
• Updated: 15/Jan/20 16:27
• Resolved: 15/Jan/20 16:27

Description

To support topology management in Cloudera Manager it'd be beneficial if Knox was able to process a descriptor that CM can generate natively. As of now, CM's CSD framework is capable of producing a file of its parameters in the following formats:

• Hadoop XML
• properties
• gflags

As the gateway-site.xml is a Hadoop XML it's quite obvious that the first option is the one that fits the most.

One XML type descriptor file may contain one or more Knox descriptors using the following structure:

• the configuration name would indicate the descriptor (topology) name
• the configuration value would list all properties of a Knox descriptor
  • service discovery related information (type, address, cluster, user/password alias)
  • services
    • name 
    • url
    • version (optional)
    • parameters (optional)
  • applications (optional)
    • name
    • parameters (optional)

A sample descriptor file would look like this:

<configuration>
  <property>
    <name>topology1</name>
    <value>
        discoveryType=ClouderaManager;
        discoveryAddress=http://host:123;
        discoveryUser=user;
        discoveryPasswordAlias=alias;
        cluster=Cluster 1;
        providerConfigRef=topology1-provider;
        app:knoxauth:param1.name=param1.value;
        app:KNOX;
        HIVE:url=http://localhost:389;
        HIVE:version=1.0;
        HIVE:httpclient.connectionTimeout=5m;
        HIVE:httpclient.socketTimeout=100m
    </value>
  </property>
  <property>
    <name>topology2</name>
    <value>
        discoveryType=ClouderaManager;
        discoveryAddress=http://host:123;
        discoveryUser=user;
        discoveryPasswordAlias=alias;
        cluster=Cluster 1;
        providerConfigRef=topology2-provider;
        app:KNOX;
        HDFS.url=https://localhost:443;
        HDFS:httpclient.connectionTimeout=5m;
        HDFS:httpclient.socketTimeout=100m
     </value>
  </property>
</configuration>
 

 

Workflow:

1. this kind of descriptor should also be placed in Knox's descriptor directory
2. once it's added or modified Knox's existing descriptor monitor should parse the XML and build one or more instance sof org.apache.knox.gateway.topology.simple.SimpleDescriptor
3. after the Java object(s) got created it (they) should be saved in the Knox descriptor directory in JSON format. As a result, the same monitor should parse the new/modified JSON descriptor(s) and re-deploys it (them) using the already existing mechanism

Issue Links

• links to: GitHub Pull Request #236