[HADOOP-16314] Make sure all end point URL is covered by the same AuthenticationFilter

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0
• Component/s: security
• Assignee: Prabhu Joseph
• Reporter: Eric Yang
• Created: 15/May/19 16:05
• Updated: 18/Mar/23 10:13
• Resolved: 05/Jun/19 22:57

Description

In the enclosed spreadsheet, it shows the list of web applications deployed by Hadoop, and filters applied to each entry point.

Hadoop web protocol impersonation has been inconsistent.  Most of entry point do not support ?doAs parameter.  This creates problem for secure gateway like Knox to proxy Hadoop web interface on behave of the end user.  When the receiving end does not check for ?doAs flag, web interface would be accessed using proxy user credential.  This can lead to all kind of security holes using path traversal to exploit Hadoop. 

In HADOOP-16287, ProxyUserAuthenticationFilter is proposed as solution to solve the web impersonation problem.  This task is to track changes required in Hadoop code base to apply authentication filter globally for each of the web service port.

Issue Links

• breaks: HDFS-14845 Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs
• breaks: HADOOP-18666 A whitelist of endpoints to skip Kerberos authentication doesn't work for ResourceManager and Job History Server
• breaks: HADOOP-16972 Ignore AuthenticationFilterInitializer for KMSWebServer
• causes: HADOOP-16356 Distcp with webhdfs is not working with ProxyUserAuthenticationFilter or AuthenticationFilter
• causes: HADOOP-16367 ApplicationHistoryServer related testcases failing
• causes: HDFS-15860 Standby Namenode bootstrap fails due to custom authentication handler being run for /imagetransfer endpoint
• is related to: HADOOP-16354 Enable AuthFilter as default for WebHdfs
• relates to: HDFS-14609 RBF: Security should use common AuthenticationFilter
• relates to: HDFS-14730 Remove unused configuration dfs.web.authentication.filter
• relates to: HDFS-15271 Remove obsolete SPNEGO configuration of NN, SNN and JN.