The current membership cache is invalidated entirely for every membership change, i.e. entries that are not affected by the change are invalidated. systems with many authorizables tend to have a full membership cache will suffer from frequent invalidation.
The way the cache is invalidated today is based on synchronous observation event. From the event alone it will be very inefficient to figure out all membership changes without extra state keeping. A more direct approach is to invalidate the membership changes directly in the cache based on the Group.addMember(), Group.removeMember() and Group.remove() methods. If the user manager is not autosave enabled, the invalidation needs to be delayed until the save call.