[SPARK-42090] Introduce sasl retry count in RetryingBlockTransferor

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.2.4, 3.3.2, 3.4.0
• Component/s: Spark Core
• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 16/Jan/23 14:00
• Updated: 24/Jan/23 18:24
• Resolved: 17/Jan/23 06:49

Description

Previously a boolean variable, saslTimeoutSeen, was used in RetryingBlockTransferor. However, the boolean variable wouldn't cover the following scenario:

1. SaslTimeoutException
2. IOException
3. SaslTimeoutException
4. IOException

Even though IOException at #2 is retried (resulting in increment of retryCount), the retryCount would be cleared at step #4.
Since the intention of saslTimeoutSeen is to undo the increment due to retrying SaslTimeoutException, we should keep a counter for SaslTimeoutException retries and subtract the value of this counter from retryCount.

Issue Links

• links to: [Github] Pull Request #39611 (tedyu)
• links to: [Github] Pull Request #39632 (akpatnam25)
• links to: [Github] Pull Request #39634 (akpatnam25)
• links to: [Github] Pull Request #39634 (akpatnam25)
• links to: [Github] Pull Request #39709 (akpatnam25)
• links to: [Github] Pull Request #39709 (akpatnam25)
• links to: [Github] Pull Request #39710 (akpatnam25)