I understand that you want a great deal of flexibility in how a web page is created, and I agree that we do not want to tie peoples' hands in saying that there is no way that they can create a new UI. But security by its nature is restrictive. Security is saying I'm sorry but you are trying to do something that is either unsafe, unwise, or beyond what you have been granted the power to do. I do not believe that this proposal will "impose mandatory significant restriction on people's freedom to create their own web UI". If you want to create a brand new UI in every way you can.
"data" : ...
The difference is that if you want to do it on a cluster that is managed by me, I have to give you approval to do that first, and install org.apache.hadoop.luke.lu.LukesWonderfulNewWidget on the proxy along with its dependencies. Also if I find out in the future that org.apache.hadoop.BobbysMaliciousWidget is doing something bad, that somehow was missed, I as the owner of this cluster can remove it from the classpath until we can either fix BobbysMaliciousWidget or determine what other actions need to be taken. Or if a critical vulnerability is found in jquery-datatable, I can upgrade jquery-datatable on the proxies that use it, and know that it is safe. I really don't know off the top of my head what steps would have to be taken if we are using JS code signing, beyond revoking all entries in the white list.
I would also like to add in that warning someone about a potentially malicious page is not secure at all. People make bad choices, especially when it is with clicking on something. I don't think that >90% of page views will be to pages served by an app master that is owned by that user. At least where I work an entire group of people own a set of map/reduce processes. Those processes are run as a user that represents those processes, not as any one user in that group. If they have to click through a warning every time they want to see a page, it will become an automatic reaction, and they will not think about it at all. I think the JS signing should handle most of this, but not all of it, especially if they modify the UI for something they specifically want.
Yes, if someone does turn off security they still have to go through the process of installing new widgets on the proxy. But this also solves the problem of displaying History Server pages in a non MR specific manor. I have not seen any proposal so far explaining how that will happen. I saw a JIRA that we should do it, but no proposal about how to do it.
Hadoop security has the design goals to be pluggable and optional. That is great but it does not mean that the Map Reduce Application Master does not have to do anything to be secure. It has to jump through all kinds of hoops so that it can be secure and be something that should be run on a secure cluster.
The code is not going to be any more complex then Hamlet is. In fact I see Hamlet being the main tool to handle all of that complexity, the difference is that instead of always writing out HTML, we could also write out JSON, or read back in the JSON. It will only take a few minor tweaks to Hamlet to allow it to the streaming.
As for Streaming limits in JSON, that is a good point, but JSON is a relatively simple protocol that is used extensively, I do not see any difficulty in adding in something like this to Jackson, if it is not there already. Simply do a check while reading in a string and if we have read more data into this string then is the limit throw an exception. Seems very straight forward. I know that everyone else is doing it is not a good logical argument but if it is such a concern then we need to go rip JSON support out of webhdfs, because it uses Jackson and is vulnerable to a malicious client. I would also like to see how ProtocolBuffers and Avro handle this. I could not find any reference to size limits on the web. In fact I saw someone saying that they have been able to send 45MB strings using ProtocolBuffers, so if there are limitations then we probably need to set them up, because I don't think they are enabled by default.
Yes, this proposal is more restrictive then allowing raw HTML/CSS/JS etc. That is because it is more secure then allowing it. Yes, it does provide a higher barrier for entry for someone who wants to write a new AppMaster and always run it on an insecure cluster, but having a GUI is not a requirement for an AppMaster, and it is no where as complex as is writing the rest of a basic AppMaster. I would like to hear what else is "fundamentally wrong on so many levels" so that I can address them as well.