The ResourceManager looks out for TaskManagers that have not had any slots allocated on them for a while, as these could be released to safe resources. If such a TM is found the RM checks via an RPC call whether the TM still holds any partitions. If no partition is held then the TM is released.
However, in the RPC callback no check is made whether the TM is actually still idle. In the meantime a slot could've been allocated on the TM.