Major Description
We are seeing lots of RSTs on our edge boxes from HTTPS-enabled clients:
20:31:42.861752 IP client-ip.56898 > www.linkedin.com.https: Flags [S], seq 2989744105, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 1060727394 ecr 0,sackOK,eol], length 0 20:31:42.861760 IP www.linkedin.com.https > client-ip.56898: Flags [S.], seq 441157858, ack 2989744106, win 14480, options [mss 1460,sackOK,TS val 579955489 ecr 1060727394,nop,wscale 7], length 0 20:31:43.005735 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack 1, win 8235, options [nop,nop,TS val 1060727553 ecr 579955489], length 0 20:31:43.012094 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq 1:216, ack 1, win 8235, options [nop,nop,TS val 1060727555 ecr 579955489], length 215 20:31:43.012115 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack 216, win 122, options [nop,nop,TS val 579955639 ecr 1060727555], length 0 20:31:43.012400 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq 1:186, ack 216, win 122, options [nop,nop,TS val 579955640 ecr 1060727555], length 185 20:31:43.157903 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack 186, win 8223, options [nop,nop,TS val 1060727703 ecr 579955640], length 0 20:31:43.157919 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq 216:222, ack 186, win 8223, options [nop,nop,TS val 1060727704 ecr 579955640], length 6 20:31:43.163988 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq 222:307, ack 186, win 8223, options [nop,nop,TS val 1060727704 ecr 579955640], length 85 20:31:43.164096 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack 307, win 122, options [nop,nop,TS val 579955791 ecr 1060727704], length 0 20:31:43.164650 IP client-ip.56898 > www.linkedin.com.https: Flags [.], seq 307:1755, ack 186, win 8223, options [nop,nop,TS val 1060727705 ecr 579955640], length 1448 20:31:43.164668 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq 1755:2472, ack 186, win 8223, options [nop,nop,TS val 1060727705 ecr 579955640], length 717 20:31:43.164677 IP www.linkedin.com.https > client-ip.56898: Flags [.], ack 2472, win 167, options [nop,nop,TS val 579955792 ecr 1060727705], length 0 20:31:43.504019 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq 186:1167, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr 1060727705], length 981 20:31:43.504043 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq 1167:1764, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr 1060727705], length 597 20:31:43.504101 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq 1764:3212, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr 1060727705], length 1448 20:31:43.504118 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq 3212:4660, ack 2472, win 167, options [nop,nop,TS val 579956131 ecr 1060727705], length 1448 20:31:43.504150 IP www.linkedin.com.https > client-ip.56898: Flags [.], seq 4660:6108, ack 2472, win 167, options [nop,nop,TS val 579956132 ecr 1060727705], length 1448 20:31:43.743945 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack 1167, win 8162, options [nop,nop,TS val 1060728286 ecr 579956131], length 0 20:31:43.743966 IP www.linkedin.com.https > client-ip.56898: Flags [P.], seq 6108:6126, ack 2472, win 167, options [nop,nop,TS val 579956371 ecr 1060728286], length 18 20:31:43.743987 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack 4660, win 7944, options [nop,nop,TS val 1060728286 ecr 579956131], length 0 20:31:43.743994 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack 6108, win 8192, options [nop,nop,TS val 1060728286 ecr 579956132], length 0 20:31:43.895889 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack 6126, win 8190, options [nop,nop,TS val 1060728437 ecr 579956371], length 0 20:31:54.189610 IP www.linkedin.com.https > client-ip.56898: Flags [F.], seq 6126, ack 2472, win 167, options [nop,nop,TS val 579966817 ecr 1060728437], length 0 20:31:54.305500 IP client-ip.56898 > www.linkedin.com.https: Flags [.], ack 6127, win 8192, options [nop,nop,TS val 1060738819 ecr 579966817], length 0 20:31:54.775695 IP client-ip.56898 > www.linkedin.com.https: Flags [P.], seq 2472:2541, ack 6127, win 8192, options [nop,nop,TS val 1060739277 ecr 579966817], length 69 20:31:54.775732 IP www.linkedin.com.https > client-ip.56898: Flags [R], seq 441163985, win 0, length 0 20:31:54.775739 IP client-ip.56898 > www.linkedin.com.https: Flags [F.], seq 2541, ack 6127, win 8192, options [nop,nop,TS val 1060739277 ecr 579966817], length 0 20:31:54.775743 IP www.linkedin.com.https > client-ip.56898: Flags [R], seq 441163985, win 0, length 0
You can see that after proxy.config.http.keep_alive_no_activity_timeout_in (10 secs) ATS is closes TCP connection without sending TLS close_notify alert which results in client sending it (last part with [P.], seq 2472:2541) and only then normally closing the connection ([F.], seq 2541), but because on ATS side socket is already closed OS replies with RST (twice actually, one for 2472:2541 and one for 2541).
From standard [RFC5246]:
Unless some other fatal alert has been transmitted, each party is required to send a close_notify alert before closing the write side of the connection. The other party MUST respond with a close_notify alert of its own and close down the connection immediately, discarding any pending writes. It is not required for the initiator of the close to wait for the responding close_notify alert before closing the read side of the connection.
[RFC5246] http://tools.ietf.org/html/rfc5246#section-7.2.1
PS. Is anyone seeing that behaviour on prod. boxes?
PPS. briang can you take a look at it?