[RANGER-1817] Audit to Solr fails to log when the number of columns are in large number

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: master, 0.7.1
• Fix Version/s: 1.0.0, master, 0.7.1
• Component/s: Ranger
• Assignee: Fatima Amjad Khan
• Reporter: Fatima Amjad Khan
• Created: 29/Sep/17 13:23
• Updated: 07/Mar/18 22:38
• Resolved: 04/Oct/17 09:42

Description

Audit to Solr fails to log when the number of columns are in large number. This is due to Solr has a hard limit on solr.StrField and if this string is exceeding max length 32766, it throws exception which causes the audit to fail. To overcome this we need to trip this in Audit records and the best place to do it is in solr schema for ranger-audits.

For this we need to change the file managed_schema in ranger and commit it to zookeeper.
Change required in the managed_schema file is, find the following in the managed_schema file and add this param to limit the length to 2500 max.

<filter class="solr.LengthFilterFactory" min="0" max="2500”/>

Final changes:

<fieldType name="key_lower_case" class="solr.TextField" sortMissingLast="true" omitNorms="true">
<analyzer>
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.LengthFilterFactory" min="0" max="2500"/>
</analyzer>
</fieldType>

Issue Links

• links to: Apache review link