[RANGER-3099] Ranger hdfs policies not syncing automatically

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 2.1.0
• Component/s: plugins, Ranger
• Environment: AWS EMR, WIndows AD
• Assignee: Unassigned
• Reporter: Anoop Kumar K M
• Created: 02/Dec/20 13:39
• Updated: 01/Sep/22 08:34

Description

Hi,

We are trying to implement Ranger 2 .1.0 on top of AWS EMR 6.1.0.

EMR 6.1.0 has  hadoop 3. The cluster is Kerberos enabled.

I have installed ranger in a separate ec2 machine and able to install hdfs plugin in EMR.

But the problem is that for policies to be applied, both ranger server and hdfs namenode should be restarted . After I restart both the policies becomes effective

Ranger admin logs shows below error.

==========

2020-11-30 10:57:42,397 [http-bio-6080-exec-9] INFO org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:345) - Request failed. loginId=null, logMessage=Unauthenticated access not allowed javax.ws.rs.WebApplicationException at org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:337) =========

 

Namenode logs show below error.

==========

 

2020-12-02 13:32:53,863 ERROR org.apache.ranger.admin.client.RangerAdminRESTClient (Thread-29): Error getting Roles; service not found. secureMode=false, user=hdfs/[e-mail] (auth:KERBEROS), response=404, serviceName=hadoopdev, lastKnownRoleVersion=-1, lastActivationTimeInMillis=1606746562885

 

2020-12-02 13:32:53,863 WARN org.apache.ranger.admin.client.RangerAdminRESTClient (Thread-29): Received 404 error code with body:[null], Ignoring
2020-12-02 13:32:53,863 INFO org.apache.ranger.admin.client.RangerAdminRESTClient (Thread-29): Skip Securetrue
2020-12-02 13:32:53,869 WARN org.apache.ranger.admin.client.RangerAdminRESTClient (Thread-29): Error getting policies. secureMode=false, user=hdfs/[e-mail] (auth:KERBEROS), response={"httpStatusCode":400,"statusCode":0}, serviceName=hadoopdev

==========

 

Under kerberos config in install.properties of ranger I have the below settings

 

--------------Kerberos Config -----------------
spnego_principal=HTTP/[e-mail]
spnego_keytab=/etc/security/keytabs/spnego.keytab
token_valid=30
cookie_domain=ip-10-98-84-189.eu-west-1.compute.internal
cookie_path=/
admin_principal=rangeradmin/[e-mail]
admin_keytab=/etc/security/keytabs/rangeradmin.keytab
lookup_principal=rangerlookup/[e-mail]
lookup_keytab=/etc/security/keytabs/rangerlookup.keytab
hadoop_conf=/etc/hadoop/conf

 

In the ranger console for the service config I have given below property

 

policy.download.auth.users = [e-mail]

 

Not sure what I am missing. Any input in this will be a great help