[YARN-2678] Improved Yarn Registry service record structure

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.6.0
• Component/s: api, resourcemanager
• Target Version/s: 2.6.0
• Assignee: Steve Loughran
• Reporter: Gour Saha
• Created: 11/Oct/14 02:18
• Updated: 01/Dec/14 03:07
• Resolved: 06/Nov/14 20:22

Description

In the process of binding to Slider AM from Slider agent python code here are some of the items I stumbled upon and would recommend as improvements.

This is how the Slider's registry looks today -

jsonservicerec{
  "description" : "Slider Application Master",
  "external" : [ {
    "api" : "org.apache.slider.appmaster",
    "addressType" : "host/port",
    "protocolType" : "hadoop/protobuf",
    "addresses" : [ [ "c6408.ambari.apache.org", "34837" ] ]
  }, {
    "api" : "org.apache.http.UI",
    "addressType" : "uri",
    "protocolType" : "webui",
    "addresses" : [ [ "http://c6408.ambari.apache.org:43314" ] ]
  }, {
    "api" : "org.apache.slider.management",
    "addressType" : "uri",
    "protocolType" : "REST",
    "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/mgmt" ] ]
  }, {
    "api" : "org.apache.slider.publisher",
    "addressType" : "uri",
    "protocolType" : "REST",
    "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/publisher" ] ]
  }, {
    "api" : "org.apache.slider.registry",
    "addressType" : "uri",
    "protocolType" : "REST",
    "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/registry" ] ]
  }, {
    "api" : "org.apache.slider.publisher.configurations",
    "addressType" : "uri",
    "protocolType" : "REST",
    "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/publisher/slider" ] ]
  } ],
  "internal" : [ {
    "api" : "org.apache.slider.agents.secure",
    "addressType" : "uri",
    "protocolType" : "REST",
    "addresses" : [ [ "https://c6408.ambari.apache.org:46958/ws/v1/slider/agents" ] ]
  }, {
    "api" : "org.apache.slider.agents.oneway",
    "addressType" : "uri",
    "protocolType" : "REST",
    "addresses" : [ [ "https://c6408.ambari.apache.org:57513/ws/v1/slider/agents" ] ]
  } ],
  "yarn:persistence" : "application",
  "yarn:id" : "application_1412974695267_0015"
}

Recommendations:
1. I would suggest to either remove the string jsonservicerec or if it is desirable to have a non-null data at all times then loop the string into the json structure as a top-level attribute to ensure that the registry data is always a valid json document.

2. The addresses attribute is currently a list of list. I would recommend to convert it to a list of dictionary objects. In the dictionary object it would be nice to have the host and port portions of objects of addressType "uri" as separate key-value pairs to avoid parsing on the client side. The URI should also be retained as a key say "uri" to avoid clients trying to generate it by concatenating host, port, resource-path, etc. Here is a proposed structure -

{
  ...
  "internal" : [ {
    "api" : "org.apache.slider.agents.secure",
    "addressType" : "uri",
    "protocolType" : "REST",
    "addresses" : [ 
       { "uri" : "https://c6408.ambari.apache.org:46958/ws/v1/slider/agents",
         "host" : "c6408.ambari.apache.org",
         "port": "46958"
       }
    ]
  } 
  ],
}

Issue Links

• is related to: SLIDER-531 adopt gour's' suggested improvements to registry