[SENTRY-1826] Create a on demand ACL based HDFS Name Node plugin

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Hdfs Plugin
• Assignee: Unassigned
• Reporter: Brian Towles
• Created: 29/Jun/17 17:39
• Updated: 29/Jun/17 17:39

Description

Since the majority of files on an HDFS system are not used all the time, a plugin could be put into place that gets the ACLs from the Sentry server only when needed. This could be done in a different plugin then the current HDFS name node plugin to allow either mechanism to be used based on the implementation choice.

This would pull ACLs from sentry on the getAclFeature call that would make a direct call to sentry for those ACLs or from a local cache if its already been retrieved once.

The current mechanism for taking updates could be replaces by a mechanism that polls for updates from Sentry and only applies/removes updates to items that are in the local cache. This would keep items in the cache fresh to changes and still only populate as is currently being used.

This would also allow us to have programmatic and/or config driven control of the size of the number of items in the cache. It could be based on size or last used timeouts for evictions. This way more frequently used ACLs are kept in cache but allow for tuning of the total amount
of space that is used to store ACLs within the Name Name itself.