Uploaded image for project: 'Qpid Dispatch'
  1. Qpid Dispatch
  2. DISPATCH-332

Heavy message loss happening with 2 interconnected routers

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Not A Bug
    • 0.6.0
    • 0.6.0
    • Routing Engine
    • None
    • Debian 8.3, Qpid Proton 0.12.2 for drivers and dependency for Qpid Dispatch, Hardware: 2 CPUs, 15 GB RAM, 30 GB HDD.

    • Important

    Description

      We are running two Dispatch Routers each configured for interior mode and the second router's configuration includes a connector to the first router with inter-router role.

      When we connect one sender to one router and one receiver to the other router both listening to the same queue, we see all messages (20,000 in our test) being transmitted.

      As soon as we start a second sender connected to the same router to which the first sender connects and sending to the same queue, we start seeing heavy message loss. Around 20% of messages are lost with each sender attempting to send 20,000 messages on its own (40,000 in total) and running in parallel with the other sender. The message loss happens regardless of the message size.

      We tried with simple_send.py, simple_recv.py as well as send and recv C executable files from Qpid Proton 0.12.2.

      We even saw a crash in the router with the following message:

      qdrouterd: /home/vsharda/qpid-dispatch/src/posix/threading.c:71: sys_mutex_lock: Assertion `result == 0' failed.
      Aborted

      The message loss was observed with the 0.6.0 SNAPSHOT taken on May 9 as well as the one taken on March 3 before the router core refactoring.

      Attachments

        1. output.txt
          419 kB
          Vishal Sharda
        2. config2.conf
          1 kB
          Vishal Sharda
        3. config1.conf
          1 kB
          Vishal Sharda

        Activity

          People

            tross Ted Ross
            vsharda Vishal Sharda
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: