[DISPATCH-1940] [http2] Router HTTP2 adaptor should pass h2spec

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.15.0
• Component/s: Protocol Adaptors
• Labels: crash
• Assignee: Ganesh Murthy
• Reporter: Jiri Daněk
• Created: 31/Jan/21 12:39
• Updated: 29/Apr/22 07:33
• Resolved: 29/Apr/22 07:33

Description

h2spec (https://github.com/summerwind/h2spec) is a HTTP/2 conformance checker, mentioned in https://blog.cloudflare.com/tools-for-debugging-testing-and-using-http-2

Running h2spec against nghttpd directly produces only one failure

Failures: 

Hypertext Transfer Protocol Version 2 (HTTP/2)
  5. Streams and Multiplexing
    5.1. Stream States
      5.1.1. Stream Identifiers
        using source address 127.0.0.1:53110
        × 2: Sends stream identifier that is numerically smaller than previous
          -> The endpoint MUST respond with a connection error of type PROTOCOL_ERROR.
             Expected: GOAWAY Frame (Error Code: PROTOCOL_ERROR)
                       Connection closed
               Actual: DATA Frame (length:147, flags:0x01, stream_id:5)

When Dispatch is put in between, it fails a few first checks, then dispatch crashes

$ wget https://github.com/summerwind/h2spec/releases/download/v2.6.0/h2spec_linux_amd64.tar.gz
$ nghttpd --no-tls -D -d /tmp 8888
$ qdrouterd -c h2spec.conf
$ ./h2spec -p 24162

I get the following result every time I run the above commands

  2. Streams and Multiplexing
    ✔ 1: Sends a PRIORITY frame on idle stream
    using source address 127.0.0.1:35814half-closed (remote) stream
    × 2: Sends a WINDOW_UPDATE frame on half-closed (remote) stream
      -> The endpoint MUST accept WINDOW_UPDATE frame.
         Expected: DATA frame
           Actual: HEADERS Frame (length:69, flags:0x04, stream_id:1)
    using source address 127.0.0.1:35816closed (remote) stream
    × 3: Sends a PRIORITY frame on half-closed (remote) stream
      -> The endpoint MUST accept PRIORITY frame.
         Expected: DATA frame
           Actual: HEADERS Frame (length:69, flags:0x04, stream_id:1)
    ✔ 4: Sends a RST_STREAM frame on half-closed (remote) stream
    ✔ 5: Sends a PRIORITY frame on closed stream

    3.8. GOAWAY
      using source address 127.0.0.1:35850
      × 1: Sends a GOAWAY frame
        -> The endpoint MUST accept GOAWAY frame.
           Expected: Connection closed
                     PING Frame (length:8, flags:0x01, stream_id:0, opaque_data:h2spec)
             Actual: Timeout

  4. HTTP Message Exchanges
    ✔ 1: Sends a GET request
    ✔ 2: Sends a HEAD request
    ✔ 3: Sends a POST request
    using source address 127.0.0.1:35866ers
    × 4: Sends a POST request with trailers
      -> The endpoint MUST respond to the request.
         Expected: HEADERS Frame (stream_id:1)
           Actual: Connection closed

  5. HPACK
    × 1: Sends a indexed header field representation

2021-01-31 13:26:25.042679 +0100 ROUTER_CORE (trace) Core action 'link_first_attach' (../src/router_core/router_core_thread.c:238)
2021-01-31 13:26:25.042912 +0100 ROUTER_CORE (info) [C31][L95] Link attached: dir=out source={(dyn)<none> expire:link} target={<none> expire:link} (../src/router_core/connections.c:1812)
2021-01-31 13:26:25.042983 +0100 HTTP_ADAPTOR (trace) [C31] Activation triggered, calling pn_raw_connection_wake() (../src/adaptors/http2/http2_adaptor.c:1524)
../src/router_core/delivery.c:109:19: runtime error: member access within null pointer of type 'struct qdr_delivery_t'
AddressSanitizer:DEADLYSIGNAL
=================================================================
==567==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000018 (pc 0x7fb3080acb9b bp 0x7fff063b3ea0 sp 0x7fff063b3e90 T0)
==567==The signal is caused by a WRITE memory access.
==567==Hint: address points to the zero page.
    #0 0x7fb3080acb9b in sys_atomic_add ../include/qpid/dispatch/atomic.h:80
    #1 0x7fb3080acc86 in sys_atomic_inc ../include/qpid/dispatch/atomic.h:209
    #2 0x7fb3080ad65f in qdr_delivery_incref ../src/router_core/delivery.c:109
    #3 0x7fb3080aeaa4 in qdr_delivery_continue ../src/router_core/delivery.c:220
    #4 0x7fb3081e69e2 in on_frame_recv_callback ../src/adaptors/http2/http2_adaptor.c:988
    #5 0x7fb307535d4d in nghttp2_session_mem_recv (/nix/store/1blnfglp53fsrd8rjmrcql18k9hdimr7-nghttp2-1.41.0-lib/lib/libnghttp2.so.14+0x10d4d)
    #6 0x7fb308200572 in handle_incoming_http ../src/adaptors/http2/http2_adaptor.c:1925
    #7 0x7fb30820998e in handle_connection_event ../src/adaptors/http2/http2_adaptor.c:2297
    #8 0x7fb3081ace0c in handle_event_with_context ../src/server.c:804
    #9 0x7fb3081ace4d in do_handle_raw_connection_event ../src/server.c:810
    #10 0x7fb3081b169c in handle ../src/server.c:1090
    #11 0x7fb3081b195d in thread_run ../src/server.c:1122
    #12 0x7fb3081b8c45 in qd_server_run ../src/server.c:1484
    #13 0x4026e4 in main_process ../router/src/main.c:113
    #14 0x404564 in main ../router/src/main.c:367
    #15 0x7fb306a1dc7c in __libc_start_main (/nix/store/9df65igwjmf2wbw0gbrrgair6piqjgmi-glibc-2.31/lib/libc.so.6+0x23c7c)
    #16 0x402419 in _start (/home/jdanek/repos/qpid/qpid-dispatch/cmake-build-debug/router/qdrouterd+0x402419)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ../include/qpid/dispatch/atomic.h:80 in sys_atomic_add
==567==ABORTING

Issue Links

• contains: DISPATCH-1896 [http2] GOAWAY frame received from server not propagated to client
• is related to: DISPATCH-1979 [http2] router crash when testing with k6 grpc performance testing tool
• links to: DISPATCH-1940 [http2] Router HTTP2 adaptor should pass h2spec #371