Uploaded image for project: 'HttpComponents HttpClient'
  1. HttpComponents HttpClient
  2. HTTPCLIENT-2116

request http website via proxy, the cookie don't work well

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 5.0.1, 5.0
    • 5.0.2, 5.1-beta1
    • HttpClient (classic)
    • None
    • operating system: macOS 10.15.6
      httpclient version: 5.0.1

    Description

      I write the code to exhibiting the problem:

      public static void main(String[] args) throws IOException {
    CookieStore cookieStore = new BasicCookieStore();
    BasicClientCookie cookie = new BasicClientCookie("test", "test");
    cookie.setDomain("cn.bing.com");
    cookie.setPath("/");
    cookieStore.addCookie(cookie);

    log.info("request no proxy: ");

    Request request = Request.get("http://cn.bing.com");
    Executor.newInstance()
            .use(cookieStore)
            .execute(request);

    log.info("request with proxy: ");

    Request requestThroughProxy = Request.get("http://cn.bing.com")
            .viaProxy("192.168.31.145:10809");
    Executor.newInstance()
            .use(cookieStore)
            .execute(requestThroughProxy);
}

      the debug log only take the org.apache.hc.client5.http.headers:

      2020-09-05 23:11:38.532  INFO 10485 --- [lication Thread] xyz.darrenluo.front.IndexController      : request no proxy: 
2020-09-05 23:11:38.762 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 >> GET / HTTP/1.1
2020-09-05 23:11:38.762 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 >> Accept-Encoding: gzip, x-gzip, deflate
2020-09-05 23:11:38.763 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 >> Host: cn.bing.com
2020-09-05 23:11:38.763 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 >> Connection: keep-alive
2020-09-05 23:11:38.763 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 >> User-Agent: Apache-HttpClient/5.0.1 (Java/14.0.2)
2020-09-05 23:11:38.763 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 >> Cookie: test=test
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << HTTP/1.1 200 OK
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Cache-Control: private, max-age=0
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Content-Length: 50197
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Content-Type: text/html; charset=utf-8
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Content-Encoding: gzip
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Vary: Accept-Encoding
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << P3P: CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND"
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Set-Cookie: SRCHD=AF=NOFORM; domain=.bing.com; expires=Mon, 05-Sep-2022 15:12:13 GMT; path=/
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Set-Cookie: SRCHUID=V=2&GUID=E44E11C37DE441DFA5940BAE884596BC&dmnchg=1; domain=.bing.com; expires=Mon, 05-Sep-2022 15:12:13 GMT; path=/
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Set-Cookie: SRCHUSR=DOB=20200905; domain=.bing.com; expires=Mon, 05-Sep-2022 15:12:13 GMT; path=/
2020-09-05 23:11:38.865 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Set-Cookie: _SS=SID=2E8C407D0B98662F2D5F4F410AB6672F; domain=.bing.com; path=/
2020-09-05 23:11:38.866 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << X-MSEdge-Ref: Ref A: BE500D3E6E4540FC8ACB3C378D68DB63 Ref B: BJ1EDGE0211 Ref C: 2020-09-05T15:12:13Z
2020-09-05 23:11:38.866 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Set-Cookie: _EDGE_S=F=1&SID=2E8C407D0B98662F2D5F4F410AB6672F; path=/; httponly; domain=bing.com
2020-09-05 23:11:38.866 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Set-Cookie: _EDGE_V=1; path=/; httponly; expires=Thu, 30-Sep-2021 15:12:13 GMT; domain=bing.com
2020-09-05 23:11:38.866 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Set-Cookie: MUID=0AFCAD618379630E0B1CA25D825762D3; samesite=none; path=/; secure; expires=Thu, 30-Sep-2021 15:12:13 GMT; domain=bing.com
2020-09-05 23:11:38.866 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Set-Cookie: MUIDB=0AFCAD618379630E0B1CA25D825762D3; path=/; httponly; expires=Thu, 30-Sep-2021 15:12:13 GMT
2020-09-05 23:11:38.866 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-0 << Date: Sat, 05 Sep 2020 15:12:13 GMT
2020-09-05 23:11:38.887  INFO 10485 --- [lication Thread] xyx.darrenluo.front.IndexController      : request with proxy: 
2020-09-05 23:11:45.729 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 >> GET http://cn.bing.com/ HTTP/1.1
2020-09-05 23:11:45.729 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 >> Accept-Encoding: gzip, x-gzip, deflate
2020-09-05 23:11:45.729 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 >> Host: cn.bing.com
2020-09-05 23:11:45.729 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 >> User-Agent: Apache-HttpClient/5.0.1 (Java/14.0.2)
2020-09-05 23:11:47.396 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << HTTP/1.1 200 OK
2020-09-05 23:11:47.396 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Content-Length: 50189
2020-09-05 23:11:47.396 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Cache-Control: private, max-age=0
2020-09-05 23:11:47.396 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Connection: keep-alive
2020-09-05 23:11:47.396 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Content-Encoding: gzip
2020-09-05 23:11:47.396 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Content-Type: text/html; charset=utf-8
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Date: Sat, 05 Sep 2020 15:12:21 GMT
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Keep-Alive: timeout=4
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << P3p: CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND"
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Proxy-Connection: keep-alive
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Set-Cookie: SRCHD=AF=NOFORM; domain=.bing.com; expires=Mon, 05-Sep-2022 15:12:21 GMT; path=/
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Set-Cookie: SRCHUID=V=2&GUID=CC885652695145E887299DF256097783&dmnchg=1; domain=.bing.com; expires=Mon, 05-Sep-2022 15:12:21 GMT; path=/
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Set-Cookie: SRCHUSR=DOB=20200905; domain=.bing.com; expires=Mon, 05-Sep-2022 15:12:21 GMT; path=/
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Set-Cookie: _SS=SID=01BC6F54CDD16E6A04936068CC5A6F68; domain=.bing.com; path=/
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Set-Cookie: _EDGE_S=F=1&SID=01BC6F54CDD16E6A04936068CC5A6F68; path=/; httponly; domain=bing.com
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Set-Cookie: _EDGE_V=1; path=/; httponly; expires=Thu, 30-Sep-2021 15:12:21 GMT; domain=bing.com
2020-09-05 23:11:47.397 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Set-Cookie: MUID=2AE7B0BA15D560010AA3BF86145E6103; samesite=none; path=/; secure; expires=Thu, 30-Sep-2021 15:12:21 GMT; domain=bing.com
2020-09-05 23:11:47.398 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Set-Cookie: MUIDB=2AE7B0BA15D560010AA3BF86145E6103; path=/; httponly; expires=Thu, 30-Sep-2021 15:12:21 GMT
2020-09-05 23:11:47.398 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << Vary: Accept-Encoding
2020-09-05 23:11:47.398 DEBUG 10485 --- [lication Thread] org.apache.hc.client5.http.headers       : http-outgoing-1 << X-Msedge-Ref: Ref A: E5AD174CC4AA4832BC8F8D750AD30CA9 Ref B: SG2EDGE1120 Ref C: 2020-09-05T15:12:21Z

      only with no proxy have 

      http-outgoing-0 >> Cookie: test=test

      and I try to fix this problem, look the github pr

      Attachments

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            darrenluo Darrenluo
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment