[RANGER-4956] RangerBasePlugin is not getting initialised when tag dedup feature is enabled

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.5.0
• Fix Version/s: 3.0.0, 2.6.0
• Component/s: plugins
• Assignee: Madhan Neethiraj
• Reporter: Anand Nadar
• Created: 10/Oct/24 12:27
• Updated: 18/Oct/24 00:17
• Resolved: 18/Oct/24 00:17

Description

When ranger.admin.supports.tags.dedup is set to true, i.e tag dedup feature is enabled, we have found that the ranger plugin is not getting initialised using the below rangerbaseplugin constructor.

public RangerBasePlugin(RangerPluginConfig pluginConfig, ServicePolicies policies, ServiceTags tags, RangerRoles roles, RangerUserStore userStore, ServiceGdsInfo gdsInfo) 

Here we have tags data as below which is provided in the above constructor

{
  "op": "add_or_update",
  "serviceName": "hive",
  "tagVersion": 7,
  "tagDefinitions": {
    "2": {
      "id": 2,
      "isEnabled": true,
      "name": "TAG2",
      "source": "Internal"
    }
  },
  "tags": {
    "10": {
      "id": 10,
      "isEnabled": true,
      "type": "TAG2"
    }
  },
  "serviceResources": [
    {
      "id": 8,
      "isEnabled": true,
      "resourceElements": {
        "database": {
          "values": [
            "db1"
          ],
          "isExcludes": false,
          "isRecursive": false
        }
      },
      "resourceSignature": "7ea12bf936f94ba73833373104ca818c249dfea758ed3366b675e86f42f01983"
    }
  ],
  "resourceToTagIds": {
    "8": [
      10
    ]
  },
  "isDelta": false,
  "tagsChangeExtent": "ALL",
  "isTagsDeduped": true,
  "id": 0
}

and therefore we set the tags in the tag enricher using the below 
tagEnricher.setServiceTags(tags);

Now in the below method
protected void setServiceTags(final ServiceTags serviceTags, final boolean rebuildOnlyIndex) 
we have this piece of code

if (!serviceTags.getIsDelta()) {					
   if (serviceTags.getIsTagsDeduped()) {						                       final int countOfDuplicateTags = serviceTags.dedupTags();
LOG.info("Number of duplicate tags removed from the received serviceTags:[" + countOfDuplicateTags + "]. Number of tags in the de-duplicated serviceTags :[" + serviceTags.getTags().size() + "].");					}					processServiceTags(serviceTags);				}

Here if serviceTags.getIsTagsDeduped() is true, we are deduping the tags again.
And when this is being triggered, an infinite for loop is triggered inserviceTags serviceTags.dedupTags() method

Below is the code which goes in infinite loop

for (Long replacerTagId = replacedIds.get(tagId); replacerTagId != null; replacerTagId = replacedIds.get(tagId)) {
                    tagId = replacerTagId;
                } 

{10, 1}

was already available in cachedTags.
Here replacedIds has {10=10}
tagId is 10

and for this as data, the the for loop goes into infinite.

Note: This behaviour is seen in the below case:
1. We initialise the plugin with the above available tag data and Rangerbaseplugin constructor. This works fine
2. Now we are trying to initialise a new Rangerbaseplugin with the same tags and this is when the plugin is not initialised and the for loop goes into an infinite state.

Suggestion:

if (!serviceTags.getIsDelta()) { 
if (serviceTags.getIsTagsDeduped()) { final int countOfDuplicateTags = serviceTags.dedupTags(); LOG.info("Number of duplicate tags removed from the received serviceTags:[" + countOfDuplicateTags + "]. Number of tags in the de-duplicated serviceTags :[" + serviceTags.getTags().size() + "]."); } processServiceTags(serviceTags); } 

Should the above piece of code in RangerTagEnricher be modified to 

if (!serviceTags.getIsDelta()) { 
processServiceTags(serviceTags); 
} 

Since tags are already deduped in ranger admin, I feel there is no need to dedup it in the plugin end.