[AMBARI-20369] Need hdfs-site for saving ranger audits to hdfs in namenode HA env

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.5.0
• Fix Version/s: 2.5.0
• Component/s: ambari-server
• Labels: Ambari
• Assignee: Mugdha Varadkar
• Reporter: Mugdha Varadkar
• Created: 09/Mar/17 09:33
• Updated: 10/Mar/17 05:33
• Resolved: 10/Mar/17 05:33

Description

For KNOX and RANGER_KMS services which supports ranger plugin, need to have hdfs-site.xml available in respective services conf directory for saving ranger audits to hdfs in namenode HA env.

Below error logs are found, if hdfs-site.xml is not available,

2017-03-01 18:48:50,150 ERROR provider.BaseAuditHandler (BaseAuditHandler.java:logError(327)) - Error writing to log file.
java.lang.IllegalArgumentException: java.net.UnknownHostException: mycluster
	at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:438)
	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:311)
	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:690)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:631)
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:160)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2795)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2829)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2811)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390)
	at org.apache.ranger.audit.destination.HDFSAuditDestination.getLogFileStream(HDFSAuditDestination.java:271)
	at org.apache.ranger.audit.destination.HDFSAuditDestination.access$000(HDFSAuditDestination.java:43)
	at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:157)
	at org.apache.ranger.audit.destination.HDFSAuditDestination$1.run(HDFSAuditDestination.java:154)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
	at org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:523)
	at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:154)
	at org.apache.ranger.audit.queue.AuditFileSpool.sendEvent(AuditFileSpool.java:880)
	at org.apache.ranger.audit.queue.AuditFileSpool.runLogAudit(AuditFileSpool.java:828)
	at org.apache.ranger.audit.queue.AuditFileSpool.run(AuditFileSpool.java:758)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: mycluster
	... 24 more
2017-03-01 18:48:50,151 ERROR queue.AuditFileSpool (AuditFileSpool.java:logError(710)) - Error sending logs to consumer. provider=knox.async.multi_dest.batch, consumer=knox.async.multi_dest.batch.hdfs{

Issue Links

• links to: Apache Review Link