Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.1.0
-
None
Description
The current code in HeartbeatReceiverSuite, executorId is set as below:
private val executorId1 = "executor-1" private val executorId2 = "executor-2"
The executorId is sent to driver when register as below:
test("expire dead hosts should kill executors with replacement (SPARK-8119)") { ... fakeSchedulerBackend.driverEndpoint.askSync[Boolean]( RegisterExecutor(executorId1, dummyExecutorEndpointRef1, "1.2.3.4", 0, Map.empty)) ... }
Receiving RegisterExecutor, the executorId will be compared with currentExecutorIdCounter as below:
case RegisterExecutor(executorId, executorRef, hostname, cores, logUrls) => if (executorDataMap.contains(executorId)) { executorRef.send(RegisterExecutorFailed("Duplicate executor ID: " + executorId)) context.reply(true) } else { ... executorDataMap.put(executorId, data) if (currentExecutorIdCounter < executorId.toInt) { currentExecutorIdCounter = executorId.toInt } ...
executorId.toInt will cause NumberformatException.
This unit test can pass currently because of askWithRetry, when catching exception, RPC will call again, thus it will go if branch and return true.