Traffic Server
  1. Traffic Server
  2. TS-506

Poor performance on proxied requests which are not cacheable.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.6
    • Component/s: Cache
    • Labels:
      None

      Description

      This is a problem similar in behavior and how to reproduce as TS-505.

      I have a test setup using lighttpd (stock config, but with the port changed to port 82, and logging disabled). I've also disabled Keep-Alive on lighttpd, with a line like this in /etc/lighttpd/lighttpd.conf

      server.max-keep-alive-requests = 0

      I've created a small file (42 bytes) in /var/www/lighttpd/cl-test.html.

      I'm running Apache TS with a stock config ("gmake install") with the following configuration changes:

      records.config:
      ---------------
      CONFIG proxy.config.http.server_port INT 80
      CONFIG proxy.config.http.cache.required_headers INT 2
      CONFIG proxy.config.log2.logging_enabled INT 0

      remap.config:
      -------------
      map http://loki.ogre.com/lighttpd/ http://localhost:82/

      I'm "benchmarking" this using "ab" from a second host, running a command like

      ab -c 20 -n 10000 http://loki.ogre.com/lighttpd/cl-test.html

      Test 1: http caching is disabled with a records.config setting

      CONFIG proxy.config.http.cache.http INT 0

      Results:

      Requests per second: 9319.75 /sec (mean)
      Time per request: 2.146 [ms] (mean)
      Time per request: 0.107 [ms] (mean, across all concurrent requests)

      Test 2: http caching enabled, records.config is

      CONFIG proxy.config.http.cache.http INT 0

      Results:

      Requests per second: 2967.59 /sec (mean)
      Time per request: 6.739 [ms] (mean)
      Time per request: 0.337 [ms] (mean, across all concurrent requests)

      So, simply enabling the cache makes it process 3x less QPS and at 3x the latency. There is again no disk I/O on either of these tests.

      If I "combine" the worst case scenario, from TS-505 and this bug, and run the test with http.cache=1 (cache enabled) and have the lighttpd server send Connection: keep-alive responses, the performance plunges:

      Requests per second: 173.09 /sec (mean)
      Time per request: 115.548 [ms] (mean)
      Time per request: 5.777 [ms] (mean, across all concurrent requests)

        Activity

        Hide
        Leif Hedstrom added a comment -

        This is resolved for now, with a change in the default configurations. Opening a separate ticket to examine when we ought to do the writer lock, but can target that for v3.1.

        Show
        Leif Hedstrom added a comment - This is resolved for now, with a change in the default configurations. Opening a separate ticket to examine when we ought to do the writer lock, but can target that for v3.1.
        Hide
        Leif Hedstrom added a comment -

        Interesting observation, but if you properly enable read-while-writer, this problem is dramatically reduced. E.g.

        CONFIG proxy.config.cache.max_doc_size INT 0
        CONFIG proxy.config.cache.enable_read_while_writer INT 1
        CONFIG proxy.config.http.background_fill_completed_threshold FLOAT 0

        Show
        Leif Hedstrom added a comment - Interesting observation, but if you properly enable read-while-writer, this problem is dramatically reduced. E.g. CONFIG proxy.config.cache.max_doc_size INT 0 CONFIG proxy.config.cache.enable_read_while_writer INT 1 CONFIG proxy.config.http.background_fill_completed_threshold FLOAT 0

          People

          • Assignee:
            Leif Hedstrom
            Reporter:
            Leif Hedstrom
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development