[KAFKA-14496] Wrong Base64 encoder used by OIDC OAuthBearerLoginCallbackHandler

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.3.1
• Fix Version/s: 3.4.0, 3.3.2
• Component/s: clients
• Assignee: Kirk True
• Reporter: Endre Vig
• Reviewer: Manikumar Reddy
• Created: 15/Dec/22 12:04
• Updated: 16/Dec/22 14:31
• Resolved: 16/Dec/22 14:31

Description

Currently our team is setting up a blueprint for our Kafka consumers/producers to provide guidelines on how to connect to our broker using the OIDC security mechanism. The blueprint is written in Java using the latest 3.3.1 Kafka library dependencies managed by Spring Boot 3.0.0.

While trying to use the new built-in org.apache.kafka.common.security.oauthbearer.secured.OAuthBearerLoginCallbackHandler introduced by KIP-768: Extend SASL/OAUTHBEARER with Support for OIDC, we've noticed that some calls to retrieve the token work out well, while some of them (seemingly randomly) are failing with 401 Unauthorized.

After some debugging we've got to the conclusion that the faulty behavior is caused by org.apache.kafka.common.security.oauthbearer.secured.HttpAccessTokenRetriever#formatAuthorizationHeader:

static String formatAuthorizationHeader(String clientId, String clientSecret) {
    clientId = sanitizeString("the token endpoint request client ID parameter", clientId);
    clientSecret = sanitizeString("the token endpoint request client secret parameter", clientSecret);
    
    String s = String.format("%s:%s", clientId, clientSecret);
    String encoded = Base64.getUrlEncoder().encodeToString(Utils.utf8(s));
    return String.format("Basic %s", encoded);
} 

The above code is using java.util.Base64#getUrlEncoder on line 311 to encode the authorization header value, which is using the alphabet described in section 5 of the RFC during the encoding algorithm. As stated by the Basic Authentication Scheme definition however, section 4 of the RFC should be used:

4. and obtains the basic-credentials by encoding this octet sequence using Base64 ([RFC4648], Section 4) into a sequence of US-ASCII characters ([RFC0020]).

The difference between the 2 alphabets are only on two characters (62: '+' vs. '-' and 63: '/' vs. '_'), that's why the 401 Unauthorized response arises only for certain credential values.

Here's a concrete example use case:

 

String s = String.format("%s:%s", "SOME_RANDOM_LONG_USER_01234", "9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
System.out.println(Base64.getUrlEncoder().encodeToString(Utils.utf8(s))); 

would print out:

U09NRV9SQU5ET01fTE9OR19VU0VSXzAxMjM0OjlRfDBgOGl-dXRlLW45a3NqTFdiXDUwIkFYQFVVRUQ1RQ== 

while

String s = String.format("%s:%s", "SOME_RANDOM_LONG_USER_01234", "9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
System.out.println(Base64.getEncoder().encodeToString(Utils.utf8(s))); 

would give:

U09NRV9SQU5ET01fTE9OR19VU0VSXzAxMjM0OjlRfDBgOGl+dXRlLW45a3NqTFdiXDUwIkFYQFVVRUQ1RQ== 

Please notice the '-' vs. '+' characters.

 

The 2 code snippets above would not behave differently for other credentials, where the encoded result doesn't use the 62nd character of the alphabet:

String s = String.format("%s:%s", "SHORT_USER_01234", "9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
System.out.println(Base64.getEncoder().encodeToString(Utils.utf8(s))); 

U0hPUlRfVVNFUl8wMTIzNDo5UXwwYDhpfnV0ZS1uOWtzakxXYlw1MCJBWEBVVUVENUU=

 

As a conclusion I would suggest that line 311 of HttpAccessTokenRetriever should be modified to use Base64.getEncoder().encodeToString(...) instead of Base64.getUrlEncoder().encodeToString(...). 

 

I'm attaching a short sample application with tests proving that the above encoding method is rejected by the standard Spring Security HTTP basic authentication as well.

Issue Links

• links to: GitHub Pull Request #13000