Running some iperf3 tests with a single router over localhost interfaces and looking at wireshark traces.
- router is receiving from the iperf3 client in 64k-byte packets
- router is sending to the iperf3 server in 512-byte packets
This is perfectly understandable if the iperf3 server connection is setting NO_DELAY=true.
This is really bad for performance in general. Network performance loves bigger data frames on the wire.
Single-router performance would be improved if dispatch coalesced the 512-byte buffers into something bigger before handing them off to proton. The idea is for TCP to share the HTTP2 raw buffering scheme that uses 16k-byte proton raw buffers.