[APEXCORE-745] Buffer server may stop processing tuples when backpressure is enabled

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.6.0
• Fix Version/s: 3.7.0
• Component/s: Buffer Server
• Assignee: Vlad Rozov
• Reporter: Vlad Rozov
• Created: 08/Jun/17 16:39
• Updated: 27/Apr/18 13:35
• Resolved: 30/Jun/17 23:28

Description

When backpressure is enabled, blocks released by a publisher are not evicted. This may lead to a condition (for example when there is a delay between publisher and subscribers request) where the publisher publishes all data blocks that the buffer server is allowed to allocate before subscribers submit subscription requests. It leads to the publisher being blocked, so it does not accept any new data and does not notify subscribers that the new data is available. At the same time subscribers are not scheduled to run after they catchup (or exit catchup), so the publisher will be blocked indefinitely. Restarting the downstream container (due to, for example, blocked downstream operator) repeat the same sequence and does not help.

Issue Links

• is related to: APEXCORE-570 Prevent upstream operators from getting too far ahead when downstream operators are slow
• links to: GitHub Pull Request #542