Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.6.0
    • Component/s: api, resourcemanager
    • Labels:
      None
    • Target Version/s:

      Description

      In the process of binding to Slider AM from Slider agent python code here are some of the items I stumbled upon and would recommend as improvements.

      This is how the Slider's registry looks today -

      jsonservicerec{
        "description" : "Slider Application Master",
        "external" : [ {
          "api" : "org.apache.slider.appmaster",
          "addressType" : "host/port",
          "protocolType" : "hadoop/protobuf",
          "addresses" : [ [ "c6408.ambari.apache.org", "34837" ] ]
        }, {
          "api" : "org.apache.http.UI",
          "addressType" : "uri",
          "protocolType" : "webui",
          "addresses" : [ [ "http://c6408.ambari.apache.org:43314" ] ]
        }, {
          "api" : "org.apache.slider.management",
          "addressType" : "uri",
          "protocolType" : "REST",
          "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/mgmt" ] ]
        }, {
          "api" : "org.apache.slider.publisher",
          "addressType" : "uri",
          "protocolType" : "REST",
          "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/publisher" ] ]
        }, {
          "api" : "org.apache.slider.registry",
          "addressType" : "uri",
          "protocolType" : "REST",
          "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/registry" ] ]
        }, {
          "api" : "org.apache.slider.publisher.configurations",
          "addressType" : "uri",
          "protocolType" : "REST",
          "addresses" : [ [ "http://c6408.ambari.apache.org:43314/ws/v1/slider/publisher/slider" ] ]
        } ],
        "internal" : [ {
          "api" : "org.apache.slider.agents.secure",
          "addressType" : "uri",
          "protocolType" : "REST",
          "addresses" : [ [ "https://c6408.ambari.apache.org:46958/ws/v1/slider/agents" ] ]
        }, {
          "api" : "org.apache.slider.agents.oneway",
          "addressType" : "uri",
          "protocolType" : "REST",
          "addresses" : [ [ "https://c6408.ambari.apache.org:57513/ws/v1/slider/agents" ] ]
        } ],
        "yarn:persistence" : "application",
        "yarn:id" : "application_1412974695267_0015"
      }
      

      Recommendations:
      1. I would suggest to either remove the string jsonservicerec or if it is desirable to have a non-null data at all times then loop the string into the json structure as a top-level attribute to ensure that the registry data is always a valid json document.

      2. The addresses attribute is currently a list of list. I would recommend to convert it to a list of dictionary objects. In the dictionary object it would be nice to have the host and port portions of objects of addressType "uri" as separate key-value pairs to avoid parsing on the client side. The URI should also be retained as a key say "uri" to avoid clients trying to generate it by concatenating host, port, resource-path, etc. Here is a proposed structure -

      {
        ...
        "internal" : [ {
          "api" : "org.apache.slider.agents.secure",
          "addressType" : "uri",
          "protocolType" : "REST",
          "addresses" : [ 
             { "uri" : "https://c6408.ambari.apache.org:46958/ws/v1/slider/agents",
               "host" : "c6408.ambari.apache.org",
               "port": "46958"
             }
          ]
        } 
        ],
      }
      

        Attachments

        1. yarnregistry.pdf
          168 kB
          Steve Loughran
        2. YARN-2678-008.patch
          70 kB
          Steve Loughran
        3. YARN-2678-007.patch
          74 kB
          Steve Loughran
        4. YARN-2678-006.patch
          69 kB
          Steve Loughran
        5. YARN-2678-003.patch
          58 kB
          Steve Loughran
        6. YARN-2678-001.patch
          53 kB
          Steve Loughran
        7. HADOOP-2678-002.patch
          58 kB
          Steve Loughran

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                gsaha Gour Saha
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: