We have a cache tier setup where cache is distributed among peers in a cluster. A given client request is typically mapped (url hashed) to one of the peers in the cluster and is cached in that particular peer. If the request is received in any other peer, it is forwarded to the peer that serves it from its cache. The cache is also disabled for that object on the peer that lands the initial request. With this setup, it's always a CACHE_MISS on the peer that lands the initial request.
ATS on the initial peer removes the conditional headers (e.g. IMS) when forwarding to the correct peer, since this is CACHE MISS. However, since, the cache is disabled on the initial peer, it should include the conditional headers letting the fwded-to peer send back a 304, instead of having to download the complete object.