This was raised by the Phoenix team. During a profiling session we noticed that catching the joinedHeap up to the current rows via seek causes a performance regression, which makes the joinedHeap only efficient when either a high or low percentage is matched by the filter.
(High is fine, because the joinedHeap will not get behind as often and does not need to be caught up, low is fine, because the seek isn't happening frequently).
In our tests we found that the solution is quite simple: Replace seek with reseek. Patch coming soon.