Major Description
Requirements:
Currently, every change to any policy causes rebuilding of policy-engine from scratch. There are several disadvantages:
1. Compute time for rebuilding
2. Large traffic from ranger-admin to each of the plugins
3. Large demand on JVM memory system resulting in frequent garbage collection and pauses of JVM.
It will be more optimal to communicate only the changes and apply them to existing policy-engine.
Design notes:
Policy changes are logged into a new database table.
Cache management in ranger-admin is enhanced to use this table to figure out changes using a previously known version number (provided by module requesting updated policies).
Policy engine supports update operation that accepts policy-deltas and returns a new policy engine with deltas applied.
Resource Trie structures are copied from older policy-engine selectively, and not rebuilt from scratch.
Backward compatibility is maintained with older plugins by adding another parameter to REST API for downloading policies.
Ranger admin as well as component plugins may be configured to optionally use policy deltas for its internal policy-engines. Policy deltas are disabled by default. In ranger-admin, policy-deltas are enabled in the ranger-admin by setting configuration variable 'ranger.admin.supports.policy.deltas' to true. In individual plugins, policy-deltas are enabled by setting configuration variable 'ranger.plugin.<service-type>.policy.rest.supports.policy.deltas' to "true".
Policy delta table is cleared of records older than a week on restart of ranger-admin.