[SPARK-2583] ConnectionManager cannot distinguish whether error occurred or not

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: Spark Core
• Target Version/s: 1.1.0
• Assignee: Josh Rosen
• Reporter: Kousuke Saruta
• Created: 18/Jul/14 17:42
• Updated: 09/Aug/14 07:26
• Resolved: 07/Aug/14 00:29

Description

ConnectionManager#handleMessage sent empty messages to another peer if some error occurred or not in onReceiveCalback.

         val ackMessage = if (onReceiveCallback != null) {
            logDebug("Calling back")
            onReceiveCallback(bufferMessage, connectionManagerId)
          } else {
            logDebug("Not calling back as callback is null")
            None
          }

          if (ackMessage.isDefined) {
            if (!ackMessage.get.isInstanceOf[BufferMessage]) {
              logDebug("Response to " + bufferMessage + " is not a buffer message, it is of type "
                + ackMessage.get.getClass)
            } else if (!ackMessage.get.asInstanceOf[BufferMessage].hasAckId) {
              logDebug("Response to " + bufferMessage + " does not have ack id set")
              ackMessage.get.asInstanceOf[BufferMessage].ackId = bufferMessage.id
            }
          }

        // We have no way to tell peer whether error occurred or not
          sendMessage(connectionManagerId, ackMessage.getOrElse {
            Message.createBufferMessage(bufferMessage.id)
          })
        }

Issue Links

• is related to: SPARK-2942 Report error messages back from server to client
• links to: [Github] Pull Request #1490 (sarutak)
• links to: [Github] Pull Request #1758 (JoshRosen)