Currently, single data structures cache is shared between several collection instances (IgniteQueue, IgniteSet).
To support iterator() and size() IgniteSet maintains plain on-heap Java sets on every node (see CacheDataStructuresManager.setDataMap). These sets duplicate backing-cache entries, both primary and backup. For big non-collocated sets it's too expensive to maintain redundant onheap data copies. The simplest way to avoid copies is to use separate cache for non-collocated IgniteSet version; hence size of set is the same as size of backing cache, and also set iterator is virtually the same as backing cache iterator.
The difference between exising set implementation and set based on separate cache should be properly documented afterwards.