[HTTPCLIENT-2113] Asynch client fails to set Host header during cross-site redirect

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.0.1
• Fix Version/s: 5.0.2
• Component/s: HttpClient (async)
• Assignee: Unassigned
• Reporter: Jesse Glick
• Created: 27/Aug/20 16:52
• Updated: 01/Sep/20 21:46
• Resolved: 01/Sep/20 21:46

Description

Run this project:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>demo</groupId>
    <artifactId>httpclientbug</artifactId>
    <version>0-SNAPSHOT</version>
    <packaging>jar</packaging>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.apache.httpcomponents.client5</groupId>
            <artifactId>httpclient5</artifactId>
            <version>5.0.1</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-jdk14</artifactId>
            <version>1.7.26</version>
            <scope>runtime</scope>
        </dependency>
    </dependencies>
</project>

import java.util.logging.ConsoleHandler;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.apache.hc.client5.http.async.methods.SimpleHttpRequests;
import org.apache.hc.client5.http.async.methods.SimpleHttpResponse;
import org.apache.hc.client5.http.classic.methods.HttpGet;
import org.apache.hc.client5.http.impl.async.CloseableHttpAsyncClient;
import org.apache.hc.client5.http.impl.async.HttpAsyncClients;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import org.apache.hc.core5.concurrent.FutureCallback;

public class Main {
    public static void main(String[] args) throws Exception {
        Logger l = Logger.getLogger("org.apache.hc.client5.http.headers");
        l.setLevel(Level.ALL);
        ConsoleHandler h = new ConsoleHandler();
        h.setLevel(Level.ALL);
        l.addHandler(h);
        String url = "https://updates.jenkins.io/download/plugins/checkmarx/2020.3.3/checkmarx.hpi";
        System.err.println("classic: " + HttpClients.createDefault().execute(new HttpGet(url)).getCode());
        CloseableHttpAsyncClient c = HttpAsyncClients.createDefault();
        c.start();
        System.err.println("asynch: " + c.execute(SimpleHttpRequests.get(url), new FutureCallback<SimpleHttpResponse>() {
            @Override
            public void completed(SimpleHttpResponse result) {}
            @Override
            public void failed(Exception x) {
                x.printStackTrace();
            }
            @Override
            public void cancelled() {}
        }).get().getCode());
    }
}

You will see that the synch client processes the two redirects (first to get.jenkins.io then to some mirror such as ftp.yz.yamagata-u.ac.jp) and successfully returns a 200 code. But the asynch client gets a 400 code after the second redirect, I believe because it is neglecting to send a Host header upon redirects (which apparently get.jenkins.io tolerates but the mirrors do not).

I had difficulty following the control flow in the code here. AsyncRedirectExec and RequestTargetHost are involved; in a debugger I could confirm that RequestTargetHost is called for all three requests in synch mode, but only for the original request in asynch mode. I suspect the issue is related to the fact that HttpAsyncClientBuilder treats RequestTargetHost specially as part of a DefaultHttpProcessor rather than being included in the HttpProcessorBuilder; for example, User-Agent from RequestUserAgent is sent on all requests.

TestRedirectExec.testCrossSiteRedirect seems like the most applicable test case but I could not follow what it was doing or where to verify concrete things like headers being set on requests.

Issue Links

• links to: jenkins-infra/usage-in-plugins #17