Charset may not be threadsafe, because the HashSet "set" is not synch.
The set is only updated by the protected add() method, which is currently only used by the protected ctors.
Perhaps add() should be private?
This would prevent any changes to the set after construction, and might be sufficient to ensure safe publication between threads.
Alternatively, make the set synch, as is done for COMMON.