[QPID-6282] low performance sending big message with RDMA transport agent

• Type: Wish
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.22
• Component/s: C++ Broker
• Labels: maxframesize qpid rdma

Environment

RHEL6.6/CentOS6.6 64bits, Mellanox ConnectX-3 dual port FDR 56Gbps IB cards, qpid packages from MRG-clone http://glitesoft.cern.ch/cern/mrg/slc6X/x86_64/RPMS/

• Assignee: Unassigned
• Reporter: An Yang
• Created: 22/Dec/14 03:25
• Updated: 22/Dec/14 03:30

Description

Transferring large messages, the message size is more than 120MB, it will cost over 250ms.

qperf -m 120m serverip，it said:
[root@node2 qpid-benchmark]# qperf -m 120m 192.168.1.1 tcp_lat
tcp_lat:
latency = 58.8 ms
[root@node2 qpid-benchmark]# qperf -m 120m 192.168.1.1 tcp_bw
tcp_bw:
bw = 2.82 GB/sec
[root@node2 qpid-benchmark]# qperf -m 120m 192.168.1.1 rc_lat
rc_lat:
latency = 20.6 ms
[root@node2 qpid-benchmark]# qperf -m 120m 192.168.1.1 rc_bw
rc_bw:
bw = 5.82 GB/sec
[root@node2 qpid-benchmark]#

I use the following cmds:
qpid-send -b 'amqp:rdma:192.168.1.1' -a 'benchmark-0;

{create:always}

' --report-total --report-header=no --timestamp=yes --sequence=no --durable False --connection-options '

{tcp-nodelay:true,reconnect:true,heartbeat:1}

' --messages 0 --content-size 120240000 --send-rate 1
qpid-receive -b 'amqp:rdma:192.168.1.1' -a 'benchmark-0;

{create:always}

' --print-content no -f --report-every 3 --receive-rate 2