[DISPATCH-1979] [http2] router crash when testing with k6 grpc performance testing tool

• Type: Test
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.16.0
• Component/s: Protocol Adaptors
• Assignee: Ganesh Murthy
• Reporter: Ganesh Murthy
• Created: 24/Feb/21 18:14
• Updated: 14/May/21 14:10
• Resolved: 17/Mar/21 14:51

Description

Use the attached router config file to start the router

download and install the k6 grpc testing tool

launch the attached javascript file using the following command -

k6 run --vus 500 --duration 120s perftest.js 

 

The router will crash with the following backtrace -

(gdb) bt
#0  0x00007fd3a5e62216 in sys_atomic_add (value=1, ref=0x18) at /home/gmurthy/opensource/qpid-dispatch/include/qpid/dispatch/atomic.h:80
#1  sys_atomic_inc (ref=0x18) at /home/gmurthy/opensource/qpid-dispatch/include/qpid/dispatch/atomic.h:209
#2  qdr_delivery_incref (delivery=delivery@entry=0x0, label=label@entry=0x7fd3a5ea4268 "qdr_delivery_continue - add to action list") at /home/gmurthy/opensource/qpid-dispatch/src/router_core/delivery.c:109
#3  0x00007fd3a5e62514 in qdr_delivery_continue (core=0x24ed6e0, in_dlv=0x0, settled=settled@entry=false) at /home/gmurthy/opensource/qpid-dispatch/src/router_core/delivery.c:220
#4  0x00007fd3a5e90a0b in on_frame_recv_callback (session=<optimized out>, frame=0x2649aa8, user_data=0x25c1f90) at /home/gmurthy/opensource/qpid-dispatch/src/adaptors/http2/http2_adaptor.c:1001
#5  0x00007fd3a599dfa1 in session_call_on_frame_received (frame=0x2649aa8, session=0x2649910) at /home/gmurthy/opensource/nghttp2/lib/nghttp2_session.c:3295
#6  session_after_header_block_received (session=0x2649910) at /home/gmurthy/opensource/nghttp2/lib/nghttp2_session.c:3796
#7  nghttp2_session_mem_recv (session=0x2649910, in=0x7fd38cb9a561 "", in@entry=0x7fd38cb9a4d4 "", inlen=429) at /home/gmurthy/opensource/nghttp2/lib/nghttp2_session.c:6230
#8  0x00007fd3a5e93d7e in handle_incoming_http (conn=0x25c1f90) at /home/gmurthy/opensource/qpid-dispatch/src/adaptors/http2/http2_adaptor.c:1934
#9  handle_connection_event (e=<optimized out>, qd_server=<optimized out>, context=0x25c1f90) at /home/gmurthy/opensource/qpid-dispatch/src/adaptors/http2/http2_adaptor.c:2337
#10 0x00007fd3a5e86961 in handle_event_with_context (context=<optimized out>, qd_server=<optimized out>, e=<optimized out>) at /home/gmurthy/opensource/qpid-dispatch/src/server.c:804
#11 do_handle_raw_connection_event (qd_server=<optimized out>, e=<optimized out>) at /home/gmurthy/opensource/qpid-dispatch/src/server.c:810
#12 handle (qd_server=qd_server@entry=0x246dd30, e=e@entry=0x24c3da0, pn_conn=pn_conn@entry=0x0, ctx=ctx@entry=0x0) at /home/gmurthy/opensource/qpid-dispatch/src/server.c:1090
#13 0x00007fd3a5e87f14 in thread_run (arg=0x246dd30) at /home/gmurthy/opensource/qpid-dispatch/src/server.c:1122
#14 0x00007fd3a5d5a3f9 in start_thread () from /lib64/libpthread.so.0
#15 0x00007fd3a58c4903 in clone () from /lib64/libc.so.6 

Issue Links

• is duplicated by: DISPATCH-2004 [http2] system_tests_grpc failure on qdr_delivery_continue
• relates to: DISPATCH-1940 [http2] Router HTTP2 adaptor should pass h2spec