[DISPATCH-825] Corrupted data on larger (>100Kb) messages

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed

Environment

Fedora 25. Master branch qpid-dispatch and qpid-cpp tools: qpid-send, qpid-receive

• Assignee: Charles E. Rolke
• Reporter: Charles E. Rolke
• Created: 12/Sep/17 20:47
• Updated: 12/Apr/19 21:43
• Resolved: 18/Sep/17 19:43

Description

Setup

Start a dispatch router with this conf file:

# Router to run qpid-interop-test
router {
    mode: interior
    id: Router.A
    workerThreads: 4
    allowUnsettledMulticast: yes
}

listener {
    host: 0.0.0.0
    port: 5672
    authenticatePeer: no
    saslMechanisms: ANONYMOUS
}

listener {
    host: localhost
    port: 5672
    authenticatePeer: no
    saslMechanisms: ANONYMOUS
}

address {
    prefix: jms.queue.qpid-interop.#
    distribution: balanced
}

log {
    module: DEFAULT
    enable: debug+
}
(

Start a receiver to receive 1000 messages:

qpid-receive -a jms.queue.qpid-interop.test --connection-options "{protocol:amqp1.0}" -m 1000 -f --print-content no --print-headers yes --ack-frequency 1

Start 1000 senders each with a different length message

#!/bin/bash
for i in `seq 100512 101511`;
        do
            qpid-send -a jms.queue.qpid-interop.test --connection-options "{protocol:amqp1.0}" -m 1 --content-size $i
        done    

Result

Eventually the receive program will exit with an error:

qpid-receive: Out of Bounds: 
  requested advance of 100552 at 42 but only 100444 available 
  (/home/chug/git/qpid-cpp/src/qpid/amqp/Decoder.cpp:307)

Observations

• Putting qd_log statements in the qd_message_send path, one at each pn_link_send() invocation, allows the setup to run the 1000 messages repeatedly. Probably it would fail eventually but in this condition it is harder to debug.
• I suspect a interlock issue between sending and receiving a single message but adding a dozen or so assert has not revealed anything yet.

Issue Links

• links to: GitHub Pull Request #194