Jonathan Eagles, thanks for working on this feature, which seems to be interesting. The benefit I can see with integrating the per-framework UI widget into the timeline server is to relieve the framework from deploying its own web server, and to prevent additional data move from the timeline server to the framework web server, and finally to the user.
To enable the web plugin, so far I can think of the following things that it's good to address.
1. Security: there're three potential issues. a) Do we want to use Hadoop http authentication to protect the pre-framework web plugin? b) We don't page-level or url matching access authorization, such that, for example, UserA can only access the web pages of it's authorized web plugin. c) Currently, everything inside timeline server belongs to YARN, such that we don't limit its access to the resources internally. Web plugin is hosted by the timeline server on behalf of the framework, and it should only have the access to the resources granted to it. For example, Tez webUI should only have access to the tez metrics in the timeline store.
2. Isolation: for some reason, the web plugin is going to crash, we should make sure it is not going to affect other components in the timeline server and other web plugins. Moreover, if multiple web plugins are hosted in the timeline server, we need to take care of their competing for the web server resources, preventing starvation.
3. Scalability: now everything is hosted in a single web server container. Hosting framework UIs will drive more traffic to the timeline server. We may want to scale up web server instances to handle users' requests. In addition, it's good think of whether we want distribute the workload according the functions: some of them serves raw REST APIs, others serve web UI of framework 1, 2, 3 and the remaining ones serve web UI of framework 4, 5, 6.