In case of multi-master Kudu cluster, the C++ client fails to re-acquire authn token in the following scenario:
- Client is running against a multi-master cluster.
- Client successfully authenticates and gets an authn token by calling ConnectToCluster.
- Client keeps the connection to leader master open, but follower masters close connections to the client due to inactivity.
- After the authn token expires, a change in the master leadership happens.
- Client tries to open a table, first making a request to the former leader master. However, the former leader returns NOT_THE_LEADER error.
Eventually, the latter operation fails out with the following error:
Timed out: GetTableSchema timed out after deadline expired
In the case of that error, along with the message above, the client is also logging messages like:
Unable to determine the new leader Master: Not authorized: Client connection negotiation failed: client connection to IP:port: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: authentication token expired