[HIVE-18766] Race condition during shutdown of RemoteDriver, error messages aren't always sent

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Spark
• Assignee: Aihua Xu
• Reporter: Sahil Takiar
• Created: 22/Feb/18 02:05
• Updated: 17/Nov/22 08:55
• Resolved: 06/Jun/18 21:16

Description

When we send an error during shutdown of the RemoteDriver, we don't wait for the error message to be sent. We just send it and then close the RPC channel. For a graceful shutdown, that doesn't seem ideal. We should at least wait a bit for the RPC message to be sent before shutting things down. I ran some tests locally and its pretty easy to hit a situation where the error message doesn't even get sent to the SparkClientImpl