Uploaded image for project: 'Qpid Proton'
  1. Qpid Proton
  2. PROTON-1167

Qpid-proton: SIGSEGV crash when a queue becomes full

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Cannot Reproduce
    • proton-0.12.0
    • None
    • proton-c
    • CentOS7 (latest)
      qpid-proton-c-0.12.0-1.el7.x86_64

    Description

      When qpid is asked to create a default queue as follows:

      qpid-config add queue foo
      

      And if an attempt is made to fill this queue to overflow with 1MB messages until we run out of space, qpid crashes as follows:

      2016-03-29 22:18:59 [Network] debug qpid.127.0.0.1:5672-127.0.0.1:43002 decoded 65536 bytes from 65536
      2016-03-29 22:18:59 [Network] debug qpid.127.0.0.1:5672-127.0.0.1:43002 decoded 1016 bytes from 1016
      2016-03-29 22:18:59 [Broker] debug received delivery: \xE4\x03\x00\x00\x00\x00\x00\x00
      2016-03-29 22:18:59 [Broker] debug Message received: 1049552 bytes
      2016-03-29 22:18:59 [System] debug Exception constructed: Maximum depth exceeded on foo: current=[count: 125, size: 103905496], max=[size: 104857600] (/builddir/build/BUILD/qpid-cpp-0.34/src/qpid/broker/Queue.cpp:1633)
      2016-03-29 22:18:59 [Network] debug qpid.127.0.0.1:5672-127.0.0.1:43002 encoded 249 bytes from 65536
      2016-03-29 22:18:59 [Network] debug qpid.127.0.0.1:5672-127.0.0.1:43002 decoded 51 bytes from 51
      2016-03-29 22:18:59 [Broker] debug received delivery: \xE4\x03\x00\x00\x00\x00\x00\x00
      2016-03-29 22:18:59 [Broker] debug Message received: 0 bytes
      2016-03-29 22:18:59 [Broker] debug clean(): 125 messages remain; head is now 0
      2016-03-29 22:18:59 [Broker] debug Message 0x69b2e0 published, state is 1 (head is now 0)
      2016-03-29 22:18:59 [Broker] debug Message 126 enqueued on foo
      
      Program received signal SIGSEGV, Segmentation fault.
      pni_process_tpwork_receiver (settle=<synthetic pointer>, delivery=0x698550, transport=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2147
      2147	  if ((int16_t) ssn->state.local_channel >= 0 && !delivery->remote.settled && delivery->state.init) {
      Missing separate debuginfos, use: debuginfo-install boost-program-options-1.53.0-25.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.13.2-10.el7.x86_64 libaio-0.3.109-13.el7.x86_64 libcom_err-1.42.9-7.el7.x86_64 libdb4-cxx-4.8.30-13.el7.x86_64 libselinux-2.2.2-6.el7.x86_64 libuuid-2.23.2-26.el7.x86_64 nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64 pcre-8.32-15.el7.x86_64 xz-libs-5.1.2-12alpha.el7.x86_64 zlib-1.2.7-15.el7.x86_64
      (gdb) bt
      #0  pni_process_tpwork_receiver (settle=<synthetic pointer>, delivery=0x698550, transport=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2147
      #1  pni_process_tpwork (transport=transport@entry=0x7fffec01c710, endpoint=<optimized out>)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2181
      #2  0x00007ffff3a898c1 in pni_process_tpwork (endpoint=<optimized out>, transport=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2164
      #3  pni_phase (phase=<optimized out>, transport=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2381
      #4  pni_process (transport=<optimized out>) at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2399
      #5  pn_output_write_amqp (transport=<optimized out>, layer=<optimized out>, bytes=0x7fffec00bf80 "", available=16384)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2550
      #6  0x00007ffff3a8aacc in transport_produce (transport=transport@entry=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2603
      #7  pn_transport_pending (transport=transport@entry=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2882
      #8  0x00007ffff3a8acd7 in pn_transport_output (transport=0x7fffec01c710, bytes=0x7fffec02f280 "", size=65536)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2630
      #9  0x00007ffff3d046ee in qpid::broker::amqp::Connection::encode (this=0x7fffec007780, buffer=0x7fffec02f280 "", size=65536)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/amqp/Connection.cpp:233
      #10 0x00007ffff749b3c4 in qpid::sys::AsynchIOHandler::idle (this=0x7fffec01ca30)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/AsynchIOHandler.cpp:221
      #11 0x00007ffff74125a6 in operator() (a0=..., this=0x7fffec000d78) at /usr/include/boost/function/function_template.hpp:767
      #12 qpid::sys::posix::AsynchIO::writeable (this=0x7fffec000b80, h=...)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/posix/AsynchIO.cpp:582
      #13 0x00007ffff749dce1 in operator() (a0=..., this=<optimized out>) at /usr/include/boost/function/function_template.hpp:767
      #14 qpid::sys::DispatchHandle::processEvent (this=0x7fffec000b88, type=qpid::sys::Poller::WRITABLE)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/DispatchHandle.cpp:283
      #15 0x00007ffff743ac4e in process (this=<synthetic pointer>, this=<synthetic pointer>)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/Poller.h:131
      #16 qpid::sys::Poller::run (this=0x659960) at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/epoll/EpollPoller.cpp:522
      #17 0x00007ffff79d54c8 in qpid::broker::Broker::run (this=0x65ca60)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/Broker.cpp:530
      #18 0x0000000000405dda in qpid::broker::QpiddBroker::execute (this=this@entry=0x7fffffffe11e, options=<optimized out>)
          at /usr/src/debug/qpid-cpp-0.34/src/posix/QpiddBroker.cpp:240
      #19 0x0000000000409d04 in qpid::broker::run_broker (argc=1, argv=0x7fffffffe4b8, hidden=<optimized out>)
          at /usr/src/debug/qpid-cpp-0.34/src/qpidd.cpp:108
      #20 0x00007ffff64e8b15 in __libc_start_main (main=0x404ce0 <main(int, char**)>, argc=1, ubp_av=0x7fffffffe4b8, 
          init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe4a8) at libc-start.c:274
      #21 0x0000000000404f51 in _start ()
      (gdb) bt full
      #0  pni_process_tpwork_receiver (settle=<synthetic pointer>, delivery=0x698550, transport=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2147
              link = 0x6987d0
              ssn = 0xf10
      #1  pni_process_tpwork (transport=transport@entry=0x7fffec01c710, endpoint=<optimized out>)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2181
              tp_next = 0x0
              settle = false
              link = <optimized out>
              dm = 0xfc0
              conn = <optimized out>
              delivery = 0x698550
      #2  0x00007ffff3a898c1 in pni_process_tpwork (endpoint=<optimized out>, transport=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2164
      No locals.
      #3  pni_phase (phase=<optimized out>, transport=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2381
              next = 0x0
              conn = <optimized out>
              endpoint = <optimized out>
      #4  pni_process (transport=<optimized out>) at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2399
              err = 0
      #5  pn_output_write_amqp (transport=<optimized out>, layer=<optimized out>, bytes=0x7fffec00bf80 "", available=16384)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2550
      No locals.
      #6  0x00007ffff3a8aacc in transport_produce (transport=transport@entry=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2603
              n = <optimized out>
              space = <optimized out>
      #7  pn_transport_pending (transport=transport@entry=0x7fffec01c710)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2882
      No locals.
      #8  0x00007ffff3a8acd7 in pn_transport_output (transport=0x7fffec01c710, bytes=0x7fffec02f280 "", size=65536)
          at /usr/src/debug/qpid-proton-0.12.0/proton-c/src/transport/transport.c:2630
              available = <optimized out>
      #9  0x00007ffff3d046ee in qpid::broker::amqp::Connection::encode (this=0x7fffec007780, buffer=0x7fffec02f280 "", size=65536)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/amqp/Connection.cpp:233
              __PRETTY_FUNCTION__ = "virtual size_t qpid::broker::amqp::Connection::encode(char*, size_t)"
              n = <optimized out>
      #10 0x00007ffff749b3c4 in qpid::sys::AsynchIOHandler::idle (this=0x7fffec01ca30)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/AsynchIOHandler.cpp:221
              encoded = <optimized out>
              buff = 0x7fffec01c910
              __PRETTY_FUNCTION__ = "void qpid::sys::AsynchIOHandler::idle(qpid::sys::AsynchIO&)"
      #11 0x00007ffff74125a6 in operator() (a0=..., this=0x7fffec000d78) at /usr/include/boost/function/function_template.hpp:767
      No locals.
      ---Type <return> to continue, or q <return> to quit---
      #12 qpid::sys::posix::AsynchIO::writeable (this=0x7fffec000b80, h=...)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/posix/AsynchIO.cpp:582
              writeStartTime = {timepoint = 6055794320274}
              total = 0
              writeCalls = 0
              __PRETTY_FUNCTION__ = "void qpid::sys::posix::AsynchIO::writeable(qpid::sys::DispatchHandle&)"
      #13 0x00007ffff749dce1 in operator() (a0=..., this=<optimized out>) at /usr/include/boost/function/function_template.hpp:767
      No locals.
      #14 qpid::sys::DispatchHandle::processEvent (this=0x7fffec000b88, type=qpid::sys::Poller::WRITABLE)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/DispatchHandle.cpp:283
              __PRETTY_FUNCTION__ = "virtual void qpid::sys::DispatchHandle::processEvent(qpid::sys::Poller::EventType)"
      #15 0x00007ffff743ac4e in process (this=<synthetic pointer>, this=<synthetic pointer>)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/Poller.h:131
      No locals.
      #16 qpid::sys::Poller::run (this=0x659960) at /usr/src/debug/qpid-cpp-0.34/src/qpid/sys/epoll/EpollPoller.cpp:522
              ss = {__val = {18446744067267100671, 18446744073709551615 <repeats 15 times>}}
              __PRETTY_FUNCTION__ = "virtual void qpid::sys::Poller::run()"
      #17 0x00007ffff79d54c8 in qpid::broker::Broker::run (this=0x65ca60)
          at /usr/src/debug/qpid-cpp-0.34/src/qpid/broker/Broker.cpp:530
              d = {<qpid::sys::Runnable> = {_vptr.Runnable = 0x7ffff7760b30 <vtable for qpid::sys::Dispatcher+16>}, poller = {
                  px = 0x659960, pn = {pi_ = 0x65d4a0}}}
              numIOThreads = 2
              t = std::vector of length 1, capacity 1 = {{impl = {px = 0x673d40, pn = {pi_ = 0x674300}}}}
              __PRETTY_FUNCTION__ = "virtual void qpid::broker::Broker::run()"
      #18 0x0000000000405dda in qpid::broker::QpiddBroker::execute (this=this@entry=0x7fffffffe11e, options=<optimized out>)
          at /usr/src/debug/qpid-cpp-0.34/src/posix/QpiddBroker.cpp:240
              brokerPtr = {px = 0x65ca60}
              __PRETTY_FUNCTION__ = "int qpid::broker::QpiddBroker::execute(qpid::broker::QpiddOptions*)"
      #19 0x0000000000409d04 in qpid::broker::run_broker (argc=1, argv=0x7fffffffe4b8, hidden=<optimized out>)
          at /usr/src/debug/qpid-cpp-0.34/src/qpidd.cpp:108
              bootOptions = {<qpid::Options> = {poOptions = {px = 0x6355f0, pn = {pi_ = 0x635660}}}, common = {<qpid::Options> = {
                    poOptions = {px = 0x635700, pn = {pi_ = 0x635770}}}, help = false, version = false, 
                  config = "/etc/qpid/qpidd.conf", clientConfig = "/etc/qpid/qpidc.conf"}, module = {<qpid::Options> = {
                    poOptions = {px = 0x636010, pn = {pi_ = 0x636080}}}, loadDir = "/usr/lib64/qpid/daemon", 
                  load = std::vector of length 0, capacity 0, noLoad = false}, log = {<qpid::Options> = {poOptions = {
                      px = 0x6366d0, pn = {pi_ = 0x636740}}}, argv0 = "/usr/sbin/qpidd", name = "Logging options", 
                  selectors = std::vector of length 1, capacity 1 = {"debug"}, deselectors = std::vector of length 0, capacity 0, 
                  time = true, level = true, thread = false, source = false, function = false, hiresTs = false, category = true, 
                  trace = false, prefix = "", sinkOptions = {_M_ptr = 0x636760}}}
              defaultPath = "/usr/lib64/qpid/daemon"
              helpArgSeen = false
              broker = {<No data fields>}
              __PRETTY_FUNCTION__ = "int qpid::broker::run_broker(int, char**, bool)"
      #20 0x00007ffff64e8b15 in __libc_start_main (main=0x404ce0 <main(int, char**)>, argc=1, ubp_av=0x7fffffffe4b8, 
          init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe4a8) at libc-start.c:274
              result = <optimized out>
      ---Type <return> to continue, or q <return> to quit---
              unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -1649862660308191144, 4214568, 140737488348336, 0, 0, 
                      1649862660174788696, 1649846041567836248}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 
                    0x40c020 <__libc_csu_init>, 0x7fffffffe4b8}, data = {prev = 0x0, cleanup = 0x0, canceltype = 4243488}}}
              not_first_call = <optimized out>
      #21 0x0000000000404f51 in _start ()
      No symbol table info available.
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            minfrin Graham Leggett
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: