Major Description
We are in the process of migrating our entire large data lakes (S3, MinIO) from resource-based to tag-based AC. This migration is phased and estimated to continue through Q3. In this phase we need to import 250K tags into Ranger from Apache Atlas. At the current, single-threaded rate of 500ms/tag, this will take 6 to 7 days.
Beyond this migration, we expect to be incorporating multiple external datasets into our datalakes at regular intervals, which will cause roughly the same quantity of tags to be imported.
So this is not a one-time thing, it is expected to recur. It would be great if tagsync could be made multi-threaded.