Massive number of ResponseImpl instances are not garbage collected after calling readEntity. By looking at VisualVM, it seems like those are indirectly referred by something stored in ThreadLocal which is used by context resolving and never been removed after all. Our production instance actually caused OutOfMemory errors after this. It might be related to https://issues.apache.org/jira/browse/CXF-6458 and looks simiar to https://java.net/jira/browse/JERSEY-2463 but not sure. Note this only happens when I used thread pool.
My workaround was bypassing readEntity() and deal with inputstream directly like following which is not quite desirable but works for now.