[NUTCH-2890] Protocol-okhttp: upgrade okhttp to 4.9.1 to address infinite connection retries

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 1.18
• Fix Version/s: 1.19
• Component/s: plugin, protocol
• Patch Info: Patch Available
• Assignee: Sebastian Nagel
• Reporter: Sebastian Nagel
• Created: 17/Sep/21 09:00
• Updated: 13/Mar/24 14:51
• Resolved: 22/Sep/21 13:05

Description

In rare situations and under heavy load (> 100 fetcher threads) I've seen that the protocol-okhttp plugin gets into a state where a significant amount of CPU time is spent in handling failed connections, see async-profiler screenshot. This is caused by okhttp#5819 which is fixed with okhttp 4.5.0).

We should upgrade to the okhttp 4.9.1 (the current stable version).

Issue Links

• links to: GitHub Pull Request #694
• links to: PR #694