This is a follow up for
TS-3714 to extend the SSL post handshake read to SPDY and H2 scenarios.
An example slow log showing very high ua_first_read.
Initially, I suspected that it might be caused by browser's connecting early before sending any bytes to TS. However, this seems to be happening too frequently and with unrealistically high delay between ua_begin and ua_first_read.
I suspect it's caused due to the code that disables the read temporarily before calling TXN_START hook and re-enables it after the API call out. The read disable is done via a 0-byte do_io_read on the client vc, but, the problem is that a valid mbuf is still passed. Based on what I am seeing, it's possible this results in actually enabling the read_vio all the way uptil ssl_read_from_net for instance (if there's a race condition and there were bytes already from the client resulting in an epoll read event), which would finally disable the read since, the ntodo (nbytes) is 0. However, this may result in the epoll event being lost until a new i/o happens from the client. I'm trying out further experiments to confirm the theory. In most cases, the read buffer already has some bytes by the time the http session and http sm is created, which makes it just work. But, if there's a slight delay in the receiving bytes after making a connection, the epoll mechanism should read it, but, due to the way the temporary read disable is being done, the event may be lost (this is coz, ATS uses the epoll edge triggered mode).
Some history on this issue -
This issue has been a problem for a long time and affects both http and https requests. When this issue was first reported, our router operations team eventually closed it indicating that disabling accept threads resolved it (Ethan Lai also reported similar observations and conclusions). It's possible that the race condition window may be slightly reduced by disabling accept threads, but, to the overall performance and through put, accept threads are very important. I now notice that the issue still exists (regardless of whether or not accept threads are enabled/disabled) and am testing further to confirm the issue.