A SASL negotiation requires a series of 1 or more challenge/responses. The current server-side RPC SASL implementation may respond with another challenge, an exception, or a switch to simple method. The server does not reply when the authentication handshake is complete.
For SASL mechanisms that require multiple exchanges before the client believes the authentication is complete, the client has an opportunity to read the exception or switch to simple. However some mechanisms, ex. PLAIN, consider the exchange complete as soon as it sends the initial response. The following proxy call will read the SASL response and throw an incomplete protobuf exception. The same issue may manifest when a client sends the final response for a multi-exchange mechanism and the server returns an exception.
Fixing the problem requires breaking RPC compatibility. We should consider having the SASL server always return success when authentication is complete.
HADOOP-8999 added a short-term workaround to send a success response only for PLAIN, and for the client to always read at least one RPC response to ensure PLAIN will work. Another complication is a SASL server returns non-null when initiating another challenge and null when authentication is established. However, the current RPC exchange does not allow a zero-byte response ("client, you initiate the exchange") to be differentiated from a null ("client, we're authenticated!"). We should consider using a different RPC status to indicate SASL authentication is in progress, so a zero-byte RPC success is interpreted as authentication is complete.