[KAFKA-14099] No REST API request logs in Kafka connect

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.4.0
• Component/s: connect
• Labels: pull-request-available
• Assignee: Alexandre Garnier
• Reporter: Alexandre Garnier
• Reviewer: Chris Egerton
• Created: 23/Jul/22 07:42
• Updated: 12/Oct/22 14:29
• Resolved: 12/Oct/22 14:29

Description

Prior to 2.2.1, when an REST API request was performed, there was a request log in the log file:

[2022-07-23 07:18:16,128] INFO 172.18.0.1 - - [23/Jul/2022:07:18:16 +0000] "GET /connectors HTTP/1.1" 200 2 "-" "curl/7.81.0" 66 (org.apache.kafka.connect.runtime.rest.RestServer:62)

Since 2.2.1, no more request logs.

 

With a bisect, I found the problem comes from PR 6651 to fix KAFKA-8304

From what I understand of the problem, the ContextHandlerCollection is added in the Server (https://github.com/dongjinleekr/kafka/blob/63a6130af30536d67fca5802005695a84c875b5e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java#L195) before handlers are really added in the ContextHandlerCollection (https://github.com/dongjinleekr/kafka/blob/63a6130af30536d67fca5802005695a84c875b5e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java#L296).
I don't know the impact on other handlers, but clearly it doesn't work for the RequestLogHandler.

 

A solution I found for the logging issue is to set the RequestLog directly in the server without using an handlers:

diff --git i/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java w/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
index ab18419efc..4d09cc0e6c 100644
--- i/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
+++ w/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/RestServer.java
@@ -187,6 +187,11 @@ public class RestServer {
     public void initializeServer() {
         log.info("Initializing REST server");
 
+        Slf4jRequestLogWriter slf4jRequestLogWriter = new Slf4jRequestLogWriter();
+        slf4jRequestLogWriter.setLoggerName(RestServer.class.getCanonicalName());
+        CustomRequestLog requestLog = new CustomRequestLog(slf4jRequestLogWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT + " %{ms}T");
+        jettyServer.setRequestLog(requestLog);
+
         /* Needed for graceful shutdown as per `setStopTimeout` documentation */
         StatisticsHandler statsHandler = new StatisticsHandler();
         statsHandler.setHandler(handlers);
@@ -275,14 +280,7 @@ public class RestServer {
             configureHttpResponsHeaderFilter(context);
         }
 
-        RequestLogHandler requestLogHandler = new RequestLogHandler();
-        Slf4jRequestLogWriter slf4jRequestLogWriter = new Slf4jRequestLogWriter();
-        slf4jRequestLogWriter.setLoggerName(RestServer.class.getCanonicalName());
-        CustomRequestLog requestLog = new CustomRequestLog(slf4jRequestLogWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT + " %{ms}T");
-        requestLogHandler.setRequestLog(requestLog);
-
         contextHandlers.add(new DefaultHandler());
-        contextHandlers.add(requestLogHandler);
 
         handlers.setHandlers(contextHandlers.toArray(new Handler[0]));
         try {

Same issue raised on StackOverflow: https://stackoverflow.com/questions/67699702/no-rest-api-logs-in-kafka-connect

Issue Links

• links to: GitHub Pull Request #12434