Details
Minor Description
TeeSinkTokenFilter can be improved in several ways, as it's written today:
The most major one is removing SinkFilter which just doesn't work and is confusing. E.g., if you set a SinkFilter which filters tokens, the attributes on the stream such as PositionIncrementAttribute are not updated. Also, if you update any attribute on the stream, you affect other SinkStreams ... It's best if we remove this confusing class, and let consumers reuse existing TokenFilters by chaining them to the sink stream.
After we do that, we can make all the cached states a single (immutable) list, which is shared between all the sink streams, so we don't need to keep many references around, and also deal with WeakReference.
Besides that there are some other minor improvements to the code that will come after we clean up this class.
From a backwards-compatibility standpoint, I don't think that SinkFilter is actually used anywhere (since it just ... confusing and doesn't work as expected), and therefore I believe it won't affect anyone. If however someone did implement a SinkFilter, it should be trivial to convert it to a TokenFilter and chain it to the SinkStream.