Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Cannot Reproduce
    • 2.2.4
    • None
    • Openwire
    • None
    • RHEL 5.2 , ActiveMQ 5.2.0

    Description

      We get this segmentation fault occasionally with a monitoring script which establishes and closes a connection to the broker.

      Here is the used script:

      import sys
      from pyactivemq import *

      broker = "tcp://127.0.0.1:61616?connection.closeTimeout=1000&connection.sendTimeout=5000&connection.socketBufferSize=128*1024"

      def check_broker():

      try:
      f = pyactivemq.ActiveMQConnectionFactory(broker)
      c = f.createConnection()
      c.start()
      c.stop()
      c.close()
      sys.exit(0)

      except Exception, e:
      sys.stderr.write("Erreur activeMQ: "+str(e))
      sys.exit(1)

      check_broker()

      Here are the full traces:

      Core was generated by `python /etc/init.d/appli_ccss.py status'.
      Program terminated with signal 11, Segmentation fault.
      #0 0x00210bf7 in apr_atomic_add32 (mem=0x87c010c, val=1) at atomic/unix/mutex.c:78
      78 apr_thread_mutex_t *mutex = hash_mutex[ATOMIC_HASH(mem)];
      (gdb) thread apply all bt full

      Thread 4 (process 13751):
      #0 0x00d97940 in __gnu_cxx::stdio_sync_filebuf<wchar_t, std::char_traits<wchar_t> >::file () from /usr/lib/libstdc++.so.6
      No symbol table info available.
      #1 0x00d99502 in std::ostream::flush () from /usr/lib/libstdc++.so.6
      No symbol table info available.
      #2 0x00d61bb1 in std::ios_base::Init::~Init () from /usr/lib/libstdc++.so.6
      No symbol table info available.
      #3 0x00d77150 in std::basic_ios<char, std::char_traits<char> >::copyfmt () from /usr/lib/libstdc++.so.6
      No symbol table info available.
      #4 0x005029c9 in exit () from /lib/libc.so.6
      No symbol table info available.
      #5 0x00707a42 in Py_Exit () from /usr/lib/libpython2.4.so.1.0
      No symbol table info available.
      #6 0x00707b12 in Py_Exit () from /usr/lib/libpython2.4.so.1.0
      No symbol table info available.
      #7 0x00707d29 in PyErr_PrintEx () from /usr/lib/libpython2.4.so.1.0
      No symbol table info available.
      #8 0x00707f5e in PyErr_Print () from /usr/lib/libpython2.4.so.1.0
      No symbol table info available.
      #9 0x00708141 in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.4.so.1.0
      No symbol table info available.
      #10 0x007088ca in PyRun_AnyFileExFlags () from /usr/lib/libpython2.4.so.1.0
      No symbol table info available.
      #11 0x0070f2d5 in Py_Main () from /usr/lib/libpython2.4.so.1.0
      No symbol table info available.
      #12 0x080485b2 in main ()
      No symbol table info available.

      Thread 3 (process 13772):
      #0 0x00434402 in __kernel_vsyscall ()
      No symbol table info available.
      #1 0x0062de1b in read () from /lib/libpthread.so.0
      No symbol table info available.
      #2 0x0020894a in apr_socket_recv (sock=0x894ccb8, buf=0x8954bc8 "", len=0xb7ad30c0) at /usr/include/bits/unistd.h:35
      rv = <value optimized out>
      arv = <value optimized out>
      #3 0x011548d9 in decaf::net::SocketInputStream::read (this=0x8947018, buffer=0x8954bc8 "", offset=0, bufferSize=1)
      at decaf/net/SocketInputStream.cpp:180
      size = 1
      result = <value optimized out>
      #4 0x01174a01 in decaf::io::BufferedInputStream::bufferData (this=0x893f240) at decaf/io/BufferedInputStream.cpp:260
      available = 0
      bytesRead = <value optimized out>
      ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, message = {
      static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      , <No data fields>},
      _M_p = 0xdc9731 "\203Ä\004[]Ã\220\220\220\220\220\220\220\220\220U\211åS\203ì\004\213E\bèèÅøÿ\201Ã\205\016\003"}},
      cause = 0x884dee8,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, _M_start = 0x13b544c, _M_finish = 0xb7ad3148,
      _M_end_of_storage = 0x11e2093}}, <No data fields>}}, <No data fields>}
      #5 0x01174f81 in decaf::io::BufferedInputStream::read (this=0x893f240, targetBuffer=0xb7ad31f4 "", offset=0,
      targetBufferSize=4) at decaf/io/BufferedInputStream.cpp:181
      bytesToCopy = 0
      totalRead = 0
      ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      , <No data fields>}, message = {
      static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      Unknown macro: {<No data fields>}

      , <No data fields>},
      _M_p = 0xffffffff <Address 0xffffffff out of bounds>}}, cause = 0x0,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      , <No data fields>}, _M_start = 0x6, _M_finish = 0x13a1860,
      _M_end_of_storage = 0x884dee8}}, <No data fields>}}, <No data fields>}
      #6 0x01178998 in decaf::io::DataInputStream::readInt (this=0x8972228) at decaf/io/DataInputStream.h:360
      buffer = "\000\000\000"
      ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, message = {
      static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      , <No data fields>},
      _M_p = 0x6 <Address 0x6 out of bounds>}}, cause = 0xb7ad325c,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, _M_start = 0xb7ad3228, _M_finish = 0x12e7780,
      _M_end_of_storage = 0x884dee8}}, <No data fields>}}, <No data fields>}
      #7 0x010e021e in activemq::connector::openwire::OpenWireFormat::unmarshal (this=0x896d8d8, dis=0x8972228)
      at activemq/connector/openwire/OpenWireFormat.cpp:227
      data = <value optimized out>
      ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      , <No data fields>}, message = {
      static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      Unknown macro: {<No data fields>}

      , <No data fields>},
      _M_p = 0xb7ad3278 "Ø2­- ï®\021\001°à\226\bèÞ\204\b`\030:\001"}}, cause = 0x10e7a2d,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      , <No data fields>}, _M_start = 0x8949d10, _M_finish = 0x884dee8,
      _M_end_of_storage = 0x13a3638}}, <No data fields>}}, <No data fields>}
      #8 0x010e5461 in activemq::connector::openwire::OpenWireCommandReader::readCommand (this=0x8949d8c)
      at activemq/connector/openwire/OpenWireCommandReader.cpp:71
      ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, message = {static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      , <No data fields>},
      _M_p = 0x0}}, cause = 0x0,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, _M_start = 0x0, _M_finish = 0x0,
      _M_end_of_storage = 0x0}}, <No data fields>}}, <No data fields>}, <No data fields>}
      #9 0x01114a9a in activemq::transport::IOTransport::run (this=0x8925f30) at activemq/transport/IOTransport.cpp:188
      command = (class activemq::transport::Command *) 0x884dee8
      #10 0x011691b8 in decaf::lang::Thread::runCallback (self=0x88e8398, param=0x8976388) at decaf/lang/Thread.cpp:125
      No locals.
      #11 0x0020e736 in dummy_worker (opaque=0x88e8398) at threadproc/unix/thread.c:142
      No locals.
      #12 0x0062745b in start_thread () from /lib/libpthread.so.0
      No symbol table info available.
      #13 0x005a7c4e in clone () from /lib/libc.so.6
      No symbol table info available.

      Thread 2 (process 13781):
      #0 0x00434402 in __kernel_vsyscall ()
      No symbol table info available.
      #1 0x0062de1b in read () from /lib/libpthread.so.0
      No symbol table info available.
      #2 0x0020894a in apr_socket_recv (sock=0x8992958, buf=0x8980260 "", len=0xb66d10c0) at /usr/include/bits/unistd.h:35
      rv = <value optimized out>
      arv = <value optimized out>
      #3 0x011548d9 in decaf::net::SocketInputStream::read (this=0x8980050, buffer=0x8980260 "", offset=0, bufferSize=1)
      at decaf/net/SocketInputStream.cpp:180
      size = 1
      result = <value optimized out>
      #4 0x01174a01 in decaf::io::BufferedInputStream::bufferData (this=0x89801a0) at decaf/io/BufferedInputStream.cpp:260
      available = 0
      bytesRead = <value optimized out>
      ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      , <No data fields>}, message = {
      static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      Unknown macro: {<No data fields>}

      , <No data fields>},
      _M_p = 0x5fd558 "double free or corruption (fasttop)"}}, cause = 0x89c4d50,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      , <No data fields>}, _M_start = 0x13a3638, _M_finish = 0x89c4d54,
      _M_end_of_storage = 0x12dc420}}, <No data fields>}}, <No data fields>}
      #5 0x01174f81 in decaf::io::BufferedInputStream::read (this=0x89801a0, targetBuffer=0xb66d11f4 "", offset=0,
      targetBufferSize=4) at decaf/io/BufferedInputStream.cpp:181
      bytesToCopy = 0
      totalRead = 0
      ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, message = {
      static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      , <No data fields>},
      _M_p = 0x89d2764 ""}}, cause = 0x13b544c,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, _M_start = 0xb66d11b8, _M_finish = 0xdfa5d4,
      _M_end_of_storage = 0x62db51}}, <No data fields>}}, <No data fields>}
      #6 0x01178998 in decaf::io::DataInputStream::readInt (this=0x89b8100) at decaf/io/DataInputStream.h:360
      buffer = "\000\000\000"
      ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      , <No data fields>}, message = {
      static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      Unknown macro: {<No data fields>}

      , <No data fields>},
      _M_p = 0x89d2760 " '\235\b"}}, cause = 0xb66d1b90,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      , <No data fields>}, _M_start = 0xb66d1b90, _M_finish = 0x2038e0,
      _M_end_of_storage = 0x899ce1c}}, <No data fields>}}, <No data fields>}
      #7 0x010e021e in activemq::connector::openwire::OpenWireFormat::unmarshal (this=0x89b3730, dis=0x89b8100)
      at activemq/connector/openwire/OpenWireFormat.cpp:227
      data = <value optimized out>
      ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, message = {
      static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      , <No data fields>},
      _M_p = 0xb66d1278 "Ø\022m¶Ç®\021\001(Í\231\bPM\234\bÈ\022m¶"}}, cause = 0x117f6cf,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, _M_start = 0x899ce18, _M_finish = 0x89c4700,
      _M_end_of_storage = 0x884deb4}}, <No data fields>}}, <No data fields>}
      #8 0x010e5461 in activemq::connector::openwire::OpenWireCommandReader::readCommand (this=0x899ee44)
      at activemq/connector/openwire/OpenWireCommandReader.cpp:71
      ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      , <No data fields>}, message = {static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      Unknown macro: {<No data fields>}

      , <No data fields>},
      _M_p = 0x0}}, cause = 0x0,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      , <No data fields>}, _M_start = 0x0, _M_finish = 0x0,
      _M_end_of_storage = 0x0}}, <No data fields>}}, <No data fields>}, <No data fields>}
      #9 0x01114a9a in activemq::transport::IOTransport::run (this=0x897fd70) at activemq/transport/IOTransport.cpp:188
      command = (class activemq::transport::Command *) 0x89c4d50
      #10 0x011691b8 in decaf::lang::Thread::runCallback (self=0x88e83d8, param=0x89bc260) at decaf/lang/Thread.cpp:125
      No locals.
      #11 0x0020e736 in dummy_worker (opaque=0x88e83d8) at threadproc/unix/thread.c:142
      No locals.
      #12 0x0062745b in start_thread () from /lib/libpthread.so.0
      No symbol table info available.
      #13 0x005a7c4e in clone () from /lib/libc.so.6
      No symbol table info available.

      Thread 1 (process 13776):
      #0 0x00210bf7 in apr_atomic_add32 (mem=0x87c010c, val=1) at atomic/unix/mutex.c:78
      old_value = 0
      mutex = <value optimized out>
      #1 0x00210c65 in apr_atomic_inc32 (mem=0x87c010c) at atomic/unix/mutex.c:132
      No locals.
      #2 0x01186460 in decaf::util::concurrent::atomic::AtomicInteger::getAndIncrement (this=0x87c0108)
      at decaf/util/concurrent/atomic/AtomicInteger.cpp:54
      No locals.
      #3 0x01119713 in activemq::transport::filters::ResponseCorrelator::oneway (this=0x87c00e8, command=0xb70d2074)
      at activemq/transport/filters/ResponseCorrelator.cpp:51
      ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, message = {static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      , <No data fields>},
      _M_p = 0x1 <Address 0x1 out of bounds>}}, cause = 0x20,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, _M_start = 0x617150, _M_finish = 0x0,
      _M_end_of_storage = 0x1}}, <No data fields>}}, <No data fields>}, <No data fields>}
      #4 0x010e7ff3 in activemq::connector::openwire::OpenWireFormatNegotiator::oneway (this=0x896e0b0, command=0xb70d2074)
      at activemq/connector/openwire/OpenWireFormatNegotiator.cpp:75
      ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      , <No data fields>}, message = {static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      Unknown macro: {<No data fields>}

      , <No data fields>},
      _M_p = 0x884ff4c "lY\234\b"}}, cause = 0x89c5944,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      , <No data fields>}, _M_start = 0x6, _M_finish = 0x10,
      _M_end_of_storage = 0x884ff48}}, <No data fields>}}, <No data fields>}, <No data fields>}
      #5 0x010e8d72 in activemq::connector::openwire::OpenWireConnector::oneway (this=0x8949d10, command=0xb70d2074)
      at activemq/connector/openwire/OpenWireConnector.cpp:1495
      ex = {<activemq::connector::ConnectorException> = {<activemq::exceptions::ActiveMQException> = {<cms::CMSException> = {<> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, <decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      , <No data fields>}, message = {static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      Unknown macro: {<No data fields>}

      , <No data fields>},
      _M_p = 0x0}}, cause = 0x6,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      , <No data fields>}, _M_start = 0x13b544c, _M_finish = 0x11e3ce0,
      _M_end_of_storage = 0x897f9b0}}, <No data fields>}}, <No data fields>}, <No data fields>}, <No data fields>}
      #6 0x010ea560 in activemq::connector::openwire::OpenWireConnector::acknowledge (this=0x8949d10, session=0x897ea78,
      consumer=0x897f870, message=0x89c5550, ackType=<value optimized out>)
      at activemq/connector/openwire/OpenWireConnector.cpp:916
      ack = {<activemq::connector::openwire::commands::BaseCommand<activemq::transport::Command>> = {<activemq::transport::Command> =

      Unknown macro: { _vptr.Command = 0x13ab268}

      , <activemq::connector::openwire::commands::BaseDataStructure> = {<activemq::connector::openwire::commands::DataStructure> = {<activemq::connector::openwire::marshal::MarshalAware> =

      Unknown macro: { _vptr.MarshalAware = 0x13ab2ec}

      , <No data fields>}, <No data fields>}, responseRequired = false, commandId = 0},
      destination = 0x884dea8, transactionId = 0x0, consumerId = 0x884dee0, ackType = 2 '\002', firstMessageId = 0x89c52a0,
      lastMessageId = 0x884ff28, messageCount = 1, static ID_MESSAGEACK = <optimized out>}
      ex = <value optimized out>
      ex = <value optimized out>
      #7 0x01077ded in activemq::core::ActiveMQConsumer::acknowledge (this=0x897f8a8, message=0x89c5608, ackType=2)
      at activemq/core/ActiveMQConsumer.cpp:458
      ex = {<cms::CMSException> = {<> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, <decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> =

      , <No data fields>}, message = {static npos = 4294967295,
      _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

      Unknown macro: {<No data fields>}

      , <No data fields>},
      _M_p = 0x1d3550 "U\211åWVS\203ì\f\213E\bè6(ÿÿ\201ÃÏš\001"}}, cause = 0x8d6f78,
      stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> =

      , <No data fields>}, _M_start = 0xb7b00c6c, _M_finish = 0x89c5ae8,
      _M_end_of_storage = 0xb70d2178}}, <No data fields>}}, <No data fields>}
      #8 0x0107825c in activemq::core::ActiveMQConsumer::afterMessageIsConsumed (this=0x897f8a8, message=0x89c5608,
      messageExpired=false) at activemq/core/ActiveMQConsumer.cpp:408
      No locals.
      #9 0x0107901f in activemq::core::ActiveMQConsumer::dispatch (this=0x897f8a8, data=@0x89c5a80)
      at activemq/core/ActiveMQConsumer.cpp:492
      message = (class activemq::core::ActiveMQMessage *) 0x89c5608
      #10 0x010921d5 in activemq::core::ActiveMQSessionExecutor::dispatch (this=0x897ed48, data=@0x89c5a80)
      at activemq/core/ActiveMQSessionExecutor.cpp:185
      lock_W = {_vptr.Lock = 0x139f470, locked = false, syncObject = 0x897ec20}
      #11 0x01093801 in activemq::core::ActiveMQSessionExecutor::dispatchAll (this=0x897ed48)
      at activemq/core/ActiveMQSessionExecutor.cpp:266
      lock_W = {_vptr.Lock = 0x139f470, locked = false, syncObject = 0x897ed60}
      dataList = {<std::_List_base<activemq::core::DispatchData,std::allocator<activemq::core::DispatchData> >> = {
      _M_impl = {<std::allocator<std::List_node<activemq::core::DispatchData> >> = {<_gnu_cxx::new_allocator<std::_List_node<activemq::core::DispatchData> >> =

      Unknown macro: {<No data fields>}

      , <No data fields>}, _M_node = {_M_next = 0x89c5a78,
      _M_prev = 0x89c5a78}}}, <No data fields>}
      lock_W = {_vptr.Lock = 0x139f470, locked = true, syncObject = 0x897ed80}
      #12 0x01093e2f in activemq::core::ActiveMQSessionExecutor::run (this=0x897ed48)
      at activemq/core/ActiveMQSessionExecutor.cpp:208
      No locals.
      #13 0x011691b8 in decaf::lang::Thread::runCallback (self=0x88e83b8, param=0x897f778) at decaf/lang/Thread.cpp:125
      No locals.
      #14 0x0020e736 in dummy_worker (opaque=0x88e83b8) at threadproc/unix/thread.c:142
      No locals.
      #15 0x0062745b in start_thread () from /lib/libpthread.so.0
      No symbol table info available.
      #16 0x005a7c4e in clone () from /lib/libc.so.6
      No symbol table info available.

      Attachments

        Activity

          Getting a backtrace from a build that doesn't have optimization turned on would be helpful. Also if you can reproduce this from a straight CPP test app that would eliminate pyactivemq from the equation.

          tabish Timothy A. Bish added a comment - Getting a backtrace from a build that doesn't have optimization turned on would be helpful. Also if you can reproduce this from a straight CPP test app that would eliminate pyactivemq from the equation.
          clayton.bluhm Clayton Bluhm added a comment -

          This issue also exists in the 2.2.5 version of ActiveMQ CPP. I am using the same server as above for the backend.

          clayton.bluhm Clayton Bluhm added a comment - This issue also exists in the 2.2.5 version of ActiveMQ CPP. I am using the same server as above for the backend.

          Hi!

          I didn't have any time these last weeks to work on this issue. I got back to this and I tried to reproduce it with activemq/apr compiled without optimization
          nor pyactivemq as you asked. I didn't success to write C++ code with the same behavior than my python program (can't get the segmentation fault).

          Anyway, I can reproduce the error using pyactivemq with the unoptimized libraries. Are you interested by the coredump in this case?

          One important detail I can give about this only process that causes the segmentation fault (which makes it different to any other process using activemq in our application):
          It is run by Linux High Availibity to monitor the state of our application every 10 seconds. To comply with the operating mode of Linux-HA, this python script will terminate
          after obtaining the application state (or by timeout), then returning status code to Linux HA. In the meantime, it will send via activeMQ a check message to the application and waits for its answer.

          The segmentation fault will occur after several hours or days, which means hundred/thousand executions of this script.
          My question is if there are any known cases of errors occuring when establishing/using/closing connection to activeMQ repeatedly instead of using persistent processes and connections?

          regards,

          Seb

          ssoilen Sebastien SOILEN added a comment - Hi! I didn't have any time these last weeks to work on this issue. I got back to this and I tried to reproduce it with activemq/apr compiled without optimization nor pyactivemq as you asked. I didn't success to write C++ code with the same behavior than my python program (can't get the segmentation fault). Anyway, I can reproduce the error using pyactivemq with the unoptimized libraries. Are you interested by the coredump in this case? One important detail I can give about this only process that causes the segmentation fault (which makes it different to any other process using activemq in our application): It is run by Linux High Availibity to monitor the state of our application every 10 seconds. To comply with the operating mode of Linux-HA, this python script will terminate after obtaining the application state (or by timeout), then returning status code to Linux HA. In the meantime, it will send via activeMQ a check message to the application and waits for its answer. The segmentation fault will occur after several hours or days, which means hundred/thousand executions of this script. My question is if there are any known cases of errors occuring when establishing/using/closing connection to activeMQ repeatedly instead of using persistent processes and connections? regards, Seb

          Currently there are some edge cases where the client can segfault on a close of a consumer, especially if in a transacted session. The code for the Consumer was rewritten in the 3.0 baseline and should be much less likely to encounter issues although there are still some deficiencies dealing with transactions.

          If you can capture a complete backtrace of all the threads when the application dumps core and attach that it would help to narrow down possibly where the error is occurring.

          tabish Timothy A. Bish added a comment - Currently there are some edge cases where the client can segfault on a close of a consumer, especially if in a transacted session. The code for the Consumer was rewritten in the 3.0 baseline and should be much less likely to encounter issues although there are still some deficiencies dealing with transactions. If you can capture a complete backtrace of all the threads when the application dumps core and attach that it would help to narrow down possibly where the error is occurring.

          Here are the traces. I hope this will help.

          Thread 1
          
          #0  0x001ebbf7 in apr_atomic_add32 (mem=0x8b720cc, val=1) at atomic/unix/mutex.c:78
                  old_value = 10895680
                  mutex = <value optimized out>
          #1  0x001ebc65 in apr_atomic_inc32 (mem=0x8b720cc) at atomic/unix/mutex.c:132
          No locals.
          #2  0x060a3340 in decaf::util::concurrent::atomic::AtomicInteger::getAndIncrement (this=0x8b720c8) at decaf/util/concurrent/atomic/AtomicInteger.cpp:54
          No locals.
          #3  0x060365f3 in activemq::transport::filters::ResponseCorrelator::oneway (this=0x8b720a8, command=0xb70b7074) at activemq/transport/filters/ResponseCorrelator.cpp:51
                  ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                  _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x1 <Address 0x1 out of bounds>}}, cause = 0x20, 
                stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                    _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xa64150, _M_finish = 0x0, 
                      _M_end_of_storage = 0x1}}, <No data fields>}}, <No data fields>}, <No data fields>}
          #4  0x06004ed3 in activemq::connector::openwire::OpenWireFormatNegotiator::oneway (this=0x8b89318, command=0xb70b7074) at activemq/connector/openwire/OpenWireFormatNegotiator.cpp:75
                  ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                  _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x8b4b84c "\234\n�\b"}}, cause = 0x8be0a74, 
                stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                    _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x6, _M_finish = 0x10, 
                      _M_end_of_storage = 0x8b4b848}}, <No data fields>}}, <No data fields>}, <No data fields>}
          #5  0x06005c52 in activemq::connector::openwire::OpenWireConnector::oneway (this=0x8b74188, command=0xb70b7074) at activemq/connector/openwire/OpenWireConnector.cpp:1495
                  ex = {<activemq::connector::ConnectorException> = {<activemq::exceptions::ActiveMQException> = {<cms::CMSException> = {<> = {<No data fields>}, <No data fields>}, <decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                    _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x0}}, cause = 0x6, 
                  stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                      _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x62d232c, _M_finish = 0x6100bc0, 
                        _M_end_of_storage = 0x8a42358}}, <No data fields>}}, <No data fields>}, <No data fields>}, <No data fields>}
          #6  0x06007440 in activemq::connector::openwire::OpenWireConnector::acknowledge (this=0x8b74188, session=0x8b99c90, consumer=0x8b9aa90, message=0x8be08f8, ackType=<value optimized out>)
              at activemq/connector/openwire/OpenWireConnector.cpp:916
                  ack = {<activemq::connector::openwire::commands::BaseCommand<activemq::transport::Command>> = {<activemq::transport::Command> = {
                _vptr.Command = 0x62c8148}, <activemq::connector::openwire::commands::BaseDataStructure> = {<activemq::connector::openwire::commands::DataStructure> = {<activemq::connector::openwire::marshal::MarshalAware> = {_vptr.MarshalAware = 0x62c81cc}, <No data fields>}, <No data fields>}, responseRequired = false, commandId = 0}, destination = 0x8a42290, transactionId = 0x0, 
            consumerId = 0x8a422c8, ackType = 2 '\002', firstMessageId = 0x8b2d2b0, lastMessageId = 0x8ad4c48, messageCount = 1, static ID_MESSAGEACK = <optimized out>}
                  ex = <value optimized out>
                  ex = <value optimized out>
          #7  0x05f94ccd in activemq::core::ActiveMQConsumer::acknowledge (this=0x8b9aae8, message=0x8be09b0, ackType=2) at activemq/core/ActiveMQConsumer.cpp:458
                  ex = {<cms::CMSException> = {<> = {<No data fields>}, <No data fields>}, <decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {
                static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x8be0be8 "�f�\bpA�"}}, 
              cause = 0xb7adac6c, 
              stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                  _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xb7ade20c, _M_finish = 0xb7adac6c, 
                    _M_end_of_storage = 0x6}}, <No data fields>}}, <No data fields>}
          #8  0x05f9513c in activemq::core::ActiveMQConsumer::afterMessageIsConsumed (this=0x8b9aae8, message=0x8be09b0, messageExpired=false) at activemq/core/ActiveMQConsumer.cpp:408
          No locals.
          #9  0x05f95eff in activemq::core::ActiveMQConsumer::dispatch (this=0x8b9aae8, data=@0x8b9aa48) at activemq/core/ActiveMQConsumer.cpp:492
                  message = (class activemq::core::ActiveMQMessage *) 0x8be09b0
          #10 0x05faf0b5 in activemq::core::ActiveMQSessionExecutor::dispatch (this=0x8b99938, data=@0x8b9aa48) at activemq/core/ActiveMQSessionExecutor.cpp:185
                  lock_W = {_vptr.Lock = 0x62bc350, locked = false, syncObject = 0x8b99ea0}
          #11 0x05fb06e1 in activemq::core::ActiveMQSessionExecutor::dispatchAll (this=0x8b99938) at activemq/core/ActiveMQSessionExecutor.cpp:266
                  lock_W = {_vptr.Lock = 0x62bc350, locked = false, syncObject = 0x8b99950}
                  dataList = {<std::_List_base<activemq::core::DispatchData,std::allocator<activemq::core::DispatchData> >> = {
              _M_impl = {<std::allocator<std::_List_node<activemq::core::DispatchData> >> = {<__gnu_cxx::new_allocator<std::_List_node<activemq::core::DispatchData> >> = {<No data fields>}, <No data fields>}, _M_node = {_M_next = 0x8b9aa40, _M_prev = 0x8b9aa40}}}, <No data fields>}
                  lock_W = {_vptr.Lock = 0x62bc350, locked = true, syncObject = 0x8b99970}
          #12 0x05fb0d0f in activemq::core::ActiveMQSessionExecutor::run (this=0x8b99938) at activemq/core/ActiveMQSessionExecutor.cpp:208
          No locals.
          #13 0x06086098 in decaf::lang::Thread::runCallback (self=0x8b43590, param=0x8b9aa28) at decaf/lang/Thread.cpp:125
          No locals.
          #14 0x001e9736 in dummy_worker (opaque=0x8b43590) at threadproc/unix/thread.c:142
          No locals.
          #15 0x00a9d45b in start_thread (arg=0xb70b7b90) at pthread_create.c:296
                  __res = <value optimized out>
                  __ignore1 = <value optimized out>
                  __ignore2 = <value optimized out>
                  pd = (struct pthread *) 0xb70b7b90
                  now = <value optimized out>
                  unwind_buf = {cancel_jmp_buf = {{jmp_buf = {11190260, 0, 4001536, -1223985992, 715026036, -1656960580}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, 
                cleanup = 0x0, canceltype = 0}}}
                  not_first_call = 0
                  robust = <value optimized out>
          #16 0x009f4e5e in clone () from /lib/libc.so.6
                  fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, 
              fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}}
                  __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xa33830
          
          
          Thread 2
          
          #0  0x00cf5402 in __kernel_vsyscall ()
          No symbol table info available.
          #1  0x00aa3e1b in read () from /lib/libpthread.so.0
          No symbol table info available.
          #2  0x001e394a in apr_socket_recv (sock=0x8badbc0, buf=0x8b9b4c8 "", len=0xb66b60c0) at /usr/include/bits/unistd.h:35
                  rv = <value optimized out>
                  arv = <value optimized out>
          #3  0x060717b9 in decaf::net::SocketInputStream::read (this=0x8b9b2b8, buffer=0x8b9b4c8 "", offset=0, bufferSize=1) at decaf/net/SocketInputStream.cpp:180
                  size = 1
                  result = <value optimized out>
          #4  0x060918e1 in decaf::io::BufferedInputStream::bufferData (this=0x8b9b408) at decaf/io/BufferedInputStream.cpp:260
                  available = 0
                  bytesRead = <value optimized out>
                  ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xa4a758 "double free or corruption (fasttop)"}}, 
              cause = 0x8b9aa40, 
              stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                  _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x62c0518, _M_finish = 0x8b9aa44, 
                    _M_end_of_storage = 0x61f9300}}, <No data fields>}}, <No data fields>}
          #5  0x06091e61 in decaf::io::BufferedInputStream::read (this=0x8b9b408, targetBuffer=0xb66b61f4 "", offset=0, targetBufferSize=4) at decaf/io/BufferedInputStream.cpp:181
                  bytesToCopy = 0
                  totalRead = 0
                  ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x8bed9dc ""}}, cause = 0x62d232c, 
              stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                  _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xb66b61b8, _M_finish = 0xe6b5d4, 
                    _M_end_of_storage = 0x8be00d8}}, <No data fields>}}, <No data fields>}
          #6  0x06095878 in decaf::io::DataInputStream::readInt (this=0x8bd3378) at decaf/io/DataInputStream.h:360
                  buffer = "\000\000\000"
                  ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x8bed9d8 "\230پ\b"}}, cause = 0xb66b6b90, 
              stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                  _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xb66b6b90, _M_finish = 0x1de8e0, 
                    _M_end_of_storage = 0x8bb8084}}, <No data fields>}}, <No data fields>}
          #7  0x05ffd0fe in activemq::connector::openwire::OpenWireFormat::unmarshal (this=0x8bce998, dis=0x8bd3378) at activemq/connector/openwire/OpenWireFormat.cpp:227
                  data = <value optimized out>
                  ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xb66b6278 "�bk��}\003\006\220\177�\b@��\b�bk�"}}, 
              cause = 0x609c5af, 
              stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                  _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x8bb8080, _M_finish = 0x8bdf700, 
                    _M_end_of_storage = 0x8be05dc}}, <No data fields>}}, <No data fields>}
          #8  0x06002341 in activemq::connector::openwire::OpenWireCommandReader::readCommand (this=0x8bba0ac) at activemq/connector/openwire/OpenWireCommandReader.cpp:71
                  ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                  _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x0}}, cause = 0x0, 
                stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                    _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x0, _M_finish = 0x0, 
                      _M_end_of_storage = 0x0}}, <No data fields>}}, <No data fields>}, <No data fields>}
          #9  0x0603197a in activemq::transport::IOTransport::run (this=0x8b9afd8) at activemq/transport/IOTransport.cpp:188
                  command = (class activemq::transport::Command *) 0x8b9aa40
          #10 0x06086098 in decaf::lang::Thread::runCallback (self=0x8b435b0, param=0x8bd74d8) at decaf/lang/Thread.cpp:125
          No locals.
          #11 0x001e9736 in dummy_worker (opaque=0x8b435b0) at threadproc/unix/thread.c:142
          No locals.
          #12 0x00a9d45b in start_thread (arg=0xb66b6b90) at pthread_create.c:296
                  __res = <value optimized out>
                  __ignore1 = <value optimized out>
                  __ignore2 = <value optimized out>
                  pd = (struct pthread *) 0xb66b6b90
                  now = <value optimized out>
                  unwind_buf = {cancel_jmp_buf = {{jmp_buf = {11190260, 0, 4001536, -1234475848, 738098804, -1656960580}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, 
                cleanup = 0x0, canceltype = 0}}}
                  not_first_call = 0
                  robust = <value optimized out>
          #13 0x009f4e5e in clone () from /lib/libc.so.6
                  fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, 
              fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}}
                  __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xa33830
          
          
          
          Thread 3
          
          #0  0x00cf5402 in __kernel_vsyscall ()
          No symbol table info available.
          #1  0x00aa3e1b in read () from /lib/libpthread.so.0
          No symbol table info available.
          #2  0x001e394a in apr_socket_recv (sock=0x8b67980, buf=0x8b6f890 "", len=0xb7ab80c0) at /usr/include/bits/unistd.h:35
                  rv = <value optimized out>
                  arv = <value optimized out>
          #3  0x060717b9 in decaf::net::SocketInputStream::read (this=0x8b5f2d0, buffer=0x8b6f890 "", offset=0, bufferSize=1) at decaf/net/SocketInputStream.cpp:180
                  size = 1
                  result = <value optimized out>
          #4  0x060918e1 in decaf::io::BufferedInputStream::bufferData (this=0x8abca00) at decaf/io/BufferedInputStream.cpp:260
                  available = 0
                  bytesRead = <value optimized out>
                  ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xe3a731 ""}}, cause = 0x8be05e8, 
              stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                  _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x62d232c, _M_finish = 0xb7ab8148, 
                    _M_end_of_storage = 0x60fef73}}, <No data fields>}}, <No data fields>}
          #5  0x06091e61 in decaf::io::BufferedInputStream::read (this=0x8abca00, targetBuffer=0xb7ab81f4 "", offset=0, targetBufferSize=4) at decaf/io/BufferedInputStream.cpp:181
                  bytesToCopy = 0
                  totalRead = 0
                  ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xffffffff <Address 0xffffffff out of bounds>}}, cause = 0x0, 
              stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                  _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x6, _M_finish = 0x62be740, 
                    _M_end_of_storage = 0x8be05e8}}, <No data fields>}}, <No data fields>}
          #6  0x06095878 in decaf::io::DataInputStream::readInt (this=0x8b8d490) at decaf/io/DataInputStream.h:360
                  buffer = "\000\000\000"
                  ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x6 <Address 0x6 out of bounds>}}, cause = 0xb7ab825c, 
              stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                  _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xb7ab8228, _M_finish = 0x6204660, 
                    _M_end_of_storage = 0x8be05e8}}, <No data fields>}}, <No data fields>}
          #7  0x05ffd0fe in activemq::connector::openwire::OpenWireFormat::unmarshal (this=0x8b88b30, dis=0x8b8d490) at activemq/connector/openwire/OpenWireFormat.cpp:227
                  data = <value optimized out>
                  ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xb7ab8278 "�\202���}\003\006\030\223�\b�\005�\b@�+\006"}}, 
              cause = 0x600490d, 
              stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                  _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x8b74188, _M_finish = 0x8be05e8, 
                    _M_end_of_storage = 0x62c0518}}, <No data fields>}}, <No data fields>}
          #8  0x06002341 in activemq::connector::openwire::OpenWireCommandReader::readCommand (this=0x8b74204) at activemq/connector/openwire/OpenWireCommandReader.cpp:71
                  ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, 
                  _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x0}}, cause = 0x0, 
                stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = {
                    _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x0, _M_finish = 0x0, 
                      _M_end_of_storage = 0x0}}, <No data fields>}}, <No data fields>}, <No data fields>}
          #9  0x0603197a in activemq::transport::IOTransport::run (this=0x8a1fce8) at activemq/transport/IOTransport.cpp:188
                  command = (class activemq::transport::Command *) 0x8be05e8
          #10 0x06086098 in decaf::lang::Thread::runCallback (self=0x8b43570, param=0x8b915f0) at decaf/lang/Thread.cpp:125
          No locals.
          #11 0x001e9736 in dummy_worker (opaque=0x8b43570) at threadproc/unix/thread.c:142
          No locals.
          #12 0x00a9d45b in start_thread (arg=0xb7ab8b90) at pthread_create.c:296
                  __res = <value optimized out>
                  __ignore1 = <value optimized out>
                  __ignore2 = <value optimized out>
                  pd = (struct pthread *) 0xb7ab8b90
                  now = <value optimized out>
                  unwind_buf = {cancel_jmp_buf = {{jmp_buf = {11190260, 0, 4001536, -1213496136, 708746868, -1656960580}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, 
                cleanup = 0x0, canceltype = 0}}}
                  not_first_call = 0
                  robust = <value optimized out>
          #13 0x009f4e5e in clone () from /lib/libc.so.6
                  fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, 
              fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}}
                  __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xa33830
          
          
          Thread 4
          
          #0  0x0098c594 in _int_free (av=0xa64140, mem=0x8b586d0) at malloc.c:4471
                  p = <value optimized out>
                  size = 48
                  fb = <value optimized out>
                  nextchunk = <value optimized out>
                  nextsize = <value optimized out>
                  prevsize = <value optimized out>
                  bck = <value optimized out>
                  fwd = <value optimized out>
                  errstr = <value optimized out>
          #1  0x009901e0 in *__GI___libc_free (mem=0x8b586d0) at malloc.c:3545
                  ar_ptr = (mstate) 0xa64140
                  p = (mchunkptr) 0x8b586c8
                  hook = (void (*)(void *, const void *)) 0
          #2  0x00e3a731 in __cxa_free_exception () from /usr/lib/libstdc++.so.6
          No symbol table info available.
          #3  0x001abc9d in std::_Rb_tree<boost::python::converter::registration, boost::python::converter::registration, std::_Identity<boost::python::converter::registration>, std::less<boost::python::converter::registration>, std::allocator<boost::python::converter::registration> >::_M_erase (this=0x1c9954, __x=0x8b53ad8) at /usr/include/c++/4.1.1/ext/new_allocator.h:94
          No locals.
          #4  0x001ab354 in __tcf_0 () at /usr/include/c++/4.1.1/bits/stl_tree.h:1264
          No locals.
          #5  0x0094f9c9 in *__GI_exit (status=0) at exit.c:75
                  atfct = <value optimized out>
                  onfct = <value optimized out>
                  cxafct = (void (*)(void *, int)) 0
                  f = (const struct exit_function * const) 0x1
          #6  0x00b54a42 in Py_Exit (sts=0) at Python/pythonrun.c:1575
          No locals.
          #7  0x00b54b12 in handle_system_exit () at Python/pythonrun.c:1041
                  exception = (PyObject *) 0xb7ee835c
                  value = (PyObject *) 0x8a14f24
                  tb = (PyObject *) 0xb7ade43c
                  exitcode = 0
          #8  0x00b54d29 in PyErr_PrintEx (set_sys_last_vars=1) at Python/pythonrun.c:1051
                  args = <value optimized out>
                  result = <value optimized out>
                  exception = <value optimized out>
                  v = <value optimized out>
                  tb = <value optimized out>
                  hook = <value optimized out>
          #9  0x00b54f5e in PyErr_Print () at Python/pythonrun.c:965
          No locals.
          #10 0x00b55141 in PyRun_SimpleFileExFlags (fp=0xbfa83a78, filename=0xbfa84b03 "./run_agent.py", closeit=1, flags=0xbfa83b74) at Python/pythonrun.c:864
                  m = <value optimized out>
                  d = (PyObject *) 0xb7efb824
                  v = (PyObject *) 0x0
                  ext = 0xbfa84b0d "t.py"
          #11 0x00b558ca in PyRun_AnyFileExFlags (fp=0x8a11008, filename=0xbfa84b03 "./run_agent.py", closeit=1, flags=0xbfa83b74) at Python/pythonrun.c:664
                  err = <value optimized out>
          #12 0x00b5c2d5 in Py_Main (argc=2, argv=0xbfa83c44) at Modules/main.c:493
                  v = <value optimized out>
                  c = <value optimized out>
                  sts = <value optimized out>
                  command = 0x0
                  filename = 0x0
                  module = 0x0
                  fp = (FILE *) 0x8a11008
                  p = <value optimized out>
                  inspect = 0
                  unbuffered = 0
                  skipfirstline = 0
                  stdin_is_interactive = 1
                  help = 0
                  version = 0
                  saw_inspect_flag = 0
                  saw_unbuffered_flag = 0
                  cf = {cf_flags = 0}
          #13 0x080485b2 in main (argc=Cannot access memory at address 0x1
          ) at Modules/python.c:23
          
          ssoilen Sebastien SOILEN added a comment - Here are the traces. I hope this will help. Thread 1 #0 0x001ebbf7 in apr_atomic_add32 (mem=0x8b720cc, val=1) at atomic/unix/mutex.c:78 old_value = 10895680 mutex = <value optimized out> #1 0x001ebc65 in apr_atomic_inc32 (mem=0x8b720cc) at atomic/unix/mutex.c:132 No locals. #2 0x060a3340 in decaf::util::concurrent::atomic::AtomicInteger::getAndIncrement (this=0x8b720c8) at decaf/util/concurrent/atomic/AtomicInteger.cpp:54 No locals. #3 0x060365f3 in activemq::transport::filters::ResponseCorrelator::oneway (this=0x8b720a8, command=0xb70b7074) at activemq/transport/filters/ResponseCorrelator.cpp:51 ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x1 <Address 0x1 out of bounds>}}, cause = 0x20, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xa64150, _M_finish = 0x0, _M_end_of_storage = 0x1}}, <No data fields>}}, <No data fields>}, <No data fields>} #4 0x06004ed3 in activemq::connector::openwire::OpenWireFormatNegotiator::oneway (this=0x8b89318, command=0xb70b7074) at activemq/connector/openwire/OpenWireFormatNegotiator.cpp:75 ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x8b4b84c "\234\n�\b"}}, cause = 0x8be0a74, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x6, _M_finish = 0x10, _M_end_of_storage = 0x8b4b848}}, <No data fields>}}, <No data fields>}, <No data fields>} #5 0x06005c52 in activemq::connector::openwire::OpenWireConnector::oneway (this=0x8b74188, command=0xb70b7074) at activemq/connector/openwire/OpenWireConnector.cpp:1495 ex = {<activemq::connector::ConnectorException> = {<activemq::exceptions::ActiveMQException> = {<cms::CMSException> = {<> = {<No data fields>}, <No data fields>}, <decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x0}}, cause = 0x6, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x62d232c, _M_finish = 0x6100bc0, _M_end_of_storage = 0x8a42358}}, <No data fields>}}, <No data fields>}, <No data fields>}, <No data fields>} #6 0x06007440 in activemq::connector::openwire::OpenWireConnector::acknowledge (this=0x8b74188, session=0x8b99c90, consumer=0x8b9aa90, message=0x8be08f8, ackType=<value optimized out>) at activemq/connector/openwire/OpenWireConnector.cpp:916 ack = {<activemq::connector::openwire::commands::BaseCommand<activemq::transport::Command>> = {<activemq::transport::Command> = { _vptr.Command = 0x62c8148}, <activemq::connector::openwire::commands::BaseDataStructure> = {<activemq::connector::openwire::commands::DataStructure> = {<activemq::connector::openwire::marshal::MarshalAware> = {_vptr.MarshalAware = 0x62c81cc}, <No data fields>}, <No data fields>}, responseRequired = false, commandId = 0}, destination = 0x8a42290, transactionId = 0x0, consumerId = 0x8a422c8, ackType = 2 '\002', firstMessageId = 0x8b2d2b0, lastMessageId = 0x8ad4c48, messageCount = 1, static ID_MESSAGEACK = <optimized out>} ex = <value optimized out> ex = <value optimized out> #7 0x05f94ccd in activemq::core::ActiveMQConsumer::acknowledge (this=0x8b9aae8, message=0x8be09b0, ackType=2) at activemq/core/ActiveMQConsumer.cpp:458 ex = {<cms::CMSException> = {<> = {<No data fields>}, <No data fields>}, <decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = { static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x8be0be8 "�f�\bpA�"}}, cause = 0xb7adac6c, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xb7ade20c, _M_finish = 0xb7adac6c, _M_end_of_storage = 0x6}}, <No data fields>}}, <No data fields>} #8 0x05f9513c in activemq::core::ActiveMQConsumer::afterMessageIsConsumed (this=0x8b9aae8, message=0x8be09b0, messageExpired=false) at activemq/core/ActiveMQConsumer.cpp:408 No locals. #9 0x05f95eff in activemq::core::ActiveMQConsumer::dispatch (this=0x8b9aae8, data=@0x8b9aa48) at activemq/core/ActiveMQConsumer.cpp:492 message = (class activemq::core::ActiveMQMessage *) 0x8be09b0 #10 0x05faf0b5 in activemq::core::ActiveMQSessionExecutor::dispatch (this=0x8b99938, data=@0x8b9aa48) at activemq/core/ActiveMQSessionExecutor.cpp:185 lock_W = {_vptr.Lock = 0x62bc350, locked = false, syncObject = 0x8b99ea0} #11 0x05fb06e1 in activemq::core::ActiveMQSessionExecutor::dispatchAll (this=0x8b99938) at activemq/core/ActiveMQSessionExecutor.cpp:266 lock_W = {_vptr.Lock = 0x62bc350, locked = false, syncObject = 0x8b99950} dataList = {<std::_List_base<activemq::core::DispatchData,std::allocator<activemq::core::DispatchData> >> = { _M_impl = {<std::allocator<std::_List_node<activemq::core::DispatchData> >> = {<__gnu_cxx::new_allocator<std::_List_node<activemq::core::DispatchData> >> = {<No data fields>}, <No data fields>}, _M_node = {_M_next = 0x8b9aa40, _M_prev = 0x8b9aa40}}}, <No data fields>} lock_W = {_vptr.Lock = 0x62bc350, locked = true, syncObject = 0x8b99970} #12 0x05fb0d0f in activemq::core::ActiveMQSessionExecutor::run (this=0x8b99938) at activemq/core/ActiveMQSessionExecutor.cpp:208 No locals. #13 0x06086098 in decaf::lang::Thread::runCallback (self=0x8b43590, param=0x8b9aa28) at decaf/lang/Thread.cpp:125 No locals. #14 0x001e9736 in dummy_worker (opaque=0x8b43590) at threadproc/unix/thread.c:142 No locals. #15 0x00a9d45b in start_thread (arg=0xb70b7b90) at pthread_create.c:296 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0xb70b7b90 now = <value optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {11190260, 0, 4001536, -1223985992, 715026036, -1656960580}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = 0 robust = <value optimized out> #16 0x009f4e5e in clone () from /lib/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xa33830 Thread 2 #0 0x00cf5402 in __kernel_vsyscall () No symbol table info available. #1 0x00aa3e1b in read () from /lib/libpthread.so.0 No symbol table info available. #2 0x001e394a in apr_socket_recv (sock=0x8badbc0, buf=0x8b9b4c8 "", len=0xb66b60c0) at /usr/include/bits/unistd.h:35 rv = <value optimized out> arv = <value optimized out> #3 0x060717b9 in decaf::net::SocketInputStream::read (this=0x8b9b2b8, buffer=0x8b9b4c8 "", offset=0, bufferSize=1) at decaf/net/SocketInputStream.cpp:180 size = 1 result = <value optimized out> #4 0x060918e1 in decaf::io::BufferedInputStream::bufferData (this=0x8b9b408) at decaf/io/BufferedInputStream.cpp:260 available = 0 bytesRead = <value optimized out> ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xa4a758 "double free or corruption (fasttop)"}}, cause = 0x8b9aa40, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x62c0518, _M_finish = 0x8b9aa44, _M_end_of_storage = 0x61f9300}}, <No data fields>}}, <No data fields>} #5 0x06091e61 in decaf::io::BufferedInputStream::read (this=0x8b9b408, targetBuffer=0xb66b61f4 "", offset=0, targetBufferSize=4) at decaf/io/BufferedInputStream.cpp:181 bytesToCopy = 0 totalRead = 0 ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x8bed9dc ""}}, cause = 0x62d232c, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xb66b61b8, _M_finish = 0xe6b5d4, _M_end_of_storage = 0x8be00d8}}, <No data fields>}}, <No data fields>} #6 0x06095878 in decaf::io::DataInputStream::readInt (this=0x8bd3378) at decaf/io/DataInputStream.h:360 buffer = "\000\000\000" ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x8bed9d8 "\230پ\b"}}, cause = 0xb66b6b90, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xb66b6b90, _M_finish = 0x1de8e0, _M_end_of_storage = 0x8bb8084}}, <No data fields>}}, <No data fields>} #7 0x05ffd0fe in activemq::connector::openwire::OpenWireFormat::unmarshal (this=0x8bce998, dis=0x8bd3378) at activemq/connector/openwire/OpenWireFormat.cpp:227 data = <value optimized out> ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xb66b6278 "�bk��}\003\006\220\177�\b@��\b�bk�"}}, cause = 0x609c5af, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x8bb8080, _M_finish = 0x8bdf700, _M_end_of_storage = 0x8be05dc}}, <No data fields>}}, <No data fields>} #8 0x06002341 in activemq::connector::openwire::OpenWireCommandReader::readCommand (this=0x8bba0ac) at activemq/connector/openwire/OpenWireCommandReader.cpp:71 ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x0}}, cause = 0x0, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x0, _M_finish = 0x0, _M_end_of_storage = 0x0}}, <No data fields>}}, <No data fields>}, <No data fields>} #9 0x0603197a in activemq::transport::IOTransport::run (this=0x8b9afd8) at activemq/transport/IOTransport.cpp:188 command = (class activemq::transport::Command *) 0x8b9aa40 #10 0x06086098 in decaf::lang::Thread::runCallback (self=0x8b435b0, param=0x8bd74d8) at decaf/lang/Thread.cpp:125 No locals. #11 0x001e9736 in dummy_worker (opaque=0x8b435b0) at threadproc/unix/thread.c:142 No locals. #12 0x00a9d45b in start_thread (arg=0xb66b6b90) at pthread_create.c:296 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0xb66b6b90 now = <value optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {11190260, 0, 4001536, -1234475848, 738098804, -1656960580}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = 0 robust = <value optimized out> #13 0x009f4e5e in clone () from /lib/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xa33830 Thread 3 #0 0x00cf5402 in __kernel_vsyscall () No symbol table info available. #1 0x00aa3e1b in read () from /lib/libpthread.so.0 No symbol table info available. #2 0x001e394a in apr_socket_recv (sock=0x8b67980, buf=0x8b6f890 "", len=0xb7ab80c0) at /usr/include/bits/unistd.h:35 rv = <value optimized out> arv = <value optimized out> #3 0x060717b9 in decaf::net::SocketInputStream::read (this=0x8b5f2d0, buffer=0x8b6f890 "", offset=0, bufferSize=1) at decaf/net/SocketInputStream.cpp:180 size = 1 result = <value optimized out> #4 0x060918e1 in decaf::io::BufferedInputStream::bufferData (this=0x8abca00) at decaf/io/BufferedInputStream.cpp:260 available = 0 bytesRead = <value optimized out> ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xe3a731 ""}}, cause = 0x8be05e8, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x62d232c, _M_finish = 0xb7ab8148, _M_end_of_storage = 0x60fef73}}, <No data fields>}}, <No data fields>} #5 0x06091e61 in decaf::io::BufferedInputStream::read (this=0x8abca00, targetBuffer=0xb7ab81f4 "", offset=0, targetBufferSize=4) at decaf/io/BufferedInputStream.cpp:181 bytesToCopy = 0 totalRead = 0 ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xffffffff <Address 0xffffffff out of bounds>}}, cause = 0x0, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x6, _M_finish = 0x62be740, _M_end_of_storage = 0x8be05e8}}, <No data fields>}}, <No data fields>} #6 0x06095878 in decaf::io::DataInputStream::readInt (this=0x8b8d490) at decaf/io/DataInputStream.h:360 buffer = "\000\000\000" ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x6 <Address 0x6 out of bounds>}}, cause = 0xb7ab825c, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0xb7ab8228, _M_finish = 0x6204660, _M_end_of_storage = 0x8be05e8}}, <No data fields>}}, <No data fields>} #7 0x05ffd0fe in activemq::connector::openwire::OpenWireFormat::unmarshal (this=0x8b88b30, dis=0x8b8d490) at activemq/connector/openwire/OpenWireFormat.cpp:227 data = <value optimized out> ex = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0xb7ab8278 "�\202���}\003\006\030\223�\b�\005�\b@�+\006"}}, cause = 0x600490d, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x8b74188, _M_finish = 0x8be05e8, _M_end_of_storage = 0x62c0518}}, <No data fields>}}, <No data fields>} #8 0x06002341 in activemq::connector::openwire::OpenWireCommandReader::readCommand (this=0x8b74204) at activemq/connector/openwire/OpenWireCommandReader.cpp:71 ex = {<decaf::io::IOException> = {<decaf::lang::Exception> = {<decaf::lang::Throwable> = {<> = {<No data fields>}, <No data fields>}, message = {static npos = 4294967295, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x0}}, cause = 0x0, stackTrace = {<std::_Vector_base<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int>,std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> > >> = { _M_impl = {<std::allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<__gnu_cxx::new_allocator<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >> = {<No data fields>}, <No data fields>}, _M_start = 0x0, _M_finish = 0x0, _M_end_of_storage = 0x0}}, <No data fields>}}, <No data fields>}, <No data fields>} #9 0x0603197a in activemq::transport::IOTransport::run (this=0x8a1fce8) at activemq/transport/IOTransport.cpp:188 command = (class activemq::transport::Command *) 0x8be05e8 #10 0x06086098 in decaf::lang::Thread::runCallback (self=0x8b43570, param=0x8b915f0) at decaf/lang/Thread.cpp:125 No locals. #11 0x001e9736 in dummy_worker (opaque=0x8b43570) at threadproc/unix/thread.c:142 No locals. #12 0x00a9d45b in start_thread (arg=0xb7ab8b90) at pthread_create.c:296 __res = <value optimized out> __ignore1 = <value optimized out> __ignore2 = <value optimized out> pd = (struct pthread *) 0xb7ab8b90 now = <value optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {11190260, 0, 4001536, -1213496136, 708746868, -1656960580}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = 0 robust = <value optimized out> #13 0x009f4e5e in clone () from /lib/libc.so.6 fstab_state = {fs_fp = 0x0, fs_buffer = 0x0, fs_mntres = {mnt_fsname = 0x0, mnt_dir = 0x0, mnt_type = 0x0, mnt_opts = 0x0, mnt_freq = 0, mnt_passno = 0}, fs_ret = {fs_spec = 0x0, fs_file = 0x0, fs_vfstype = 0x0, fs_mntops = 0x0, fs_type = 0x0, fs_freq = 0, fs_passno = 0}} __elf_set___libc_subfreeres_element_fstab_free__ = (const void *) 0xa33830 Thread 4 #0 0x0098c594 in _int_free (av=0xa64140, mem=0x8b586d0) at malloc.c:4471 p = <value optimized out> size = 48 fb = <value optimized out> nextchunk = <value optimized out> nextsize = <value optimized out> prevsize = <value optimized out> bck = <value optimized out> fwd = <value optimized out> errstr = <value optimized out> #1 0x009901e0 in *__GI___libc_free (mem=0x8b586d0) at malloc.c:3545 ar_ptr = (mstate) 0xa64140 p = (mchunkptr) 0x8b586c8 hook = (void (*)(void *, const void *)) 0 #2 0x00e3a731 in __cxa_free_exception () from /usr/lib/libstdc++.so.6 No symbol table info available. #3 0x001abc9d in std::_Rb_tree<boost::python::converter::registration, boost::python::converter::registration, std::_Identity<boost::python::converter::registration>, std::less<boost::python::converter::registration>, std::allocator<boost::python::converter::registration> >::_M_erase (this=0x1c9954, __x=0x8b53ad8) at /usr/include/c++/4.1.1/ext/new_allocator.h:94 No locals. #4 0x001ab354 in __tcf_0 () at /usr/include/c++/4.1.1/bits/stl_tree.h:1264 No locals. #5 0x0094f9c9 in *__GI_exit (status=0) at exit.c:75 atfct = <value optimized out> onfct = <value optimized out> cxafct = (void (*)(void *, int)) 0 f = (const struct exit_function * const) 0x1 #6 0x00b54a42 in Py_Exit (sts=0) at Python/pythonrun.c:1575 No locals. #7 0x00b54b12 in handle_system_exit () at Python/pythonrun.c:1041 exception = (PyObject *) 0xb7ee835c value = (PyObject *) 0x8a14f24 tb = (PyObject *) 0xb7ade43c exitcode = 0 #8 0x00b54d29 in PyErr_PrintEx (set_sys_last_vars=1) at Python/pythonrun.c:1051 args = <value optimized out> result = <value optimized out> exception = <value optimized out> v = <value optimized out> tb = <value optimized out> hook = <value optimized out> #9 0x00b54f5e in PyErr_Print () at Python/pythonrun.c:965 No locals. #10 0x00b55141 in PyRun_SimpleFileExFlags (fp=0xbfa83a78, filename=0xbfa84b03 "./run_agent.py", closeit=1, flags=0xbfa83b74) at Python/pythonrun.c:864 m = <value optimized out> d = (PyObject *) 0xb7efb824 v = (PyObject *) 0x0 ext = 0xbfa84b0d "t.py" #11 0x00b558ca in PyRun_AnyFileExFlags (fp=0x8a11008, filename=0xbfa84b03 "./run_agent.py", closeit=1, flags=0xbfa83b74) at Python/pythonrun.c:664 err = <value optimized out> #12 0x00b5c2d5 in Py_Main (argc=2, argv=0xbfa83c44) at Modules/main.c:493 v = <value optimized out> c = <value optimized out> sts = <value optimized out> command = 0x0 filename = 0x0 module = 0x0 fp = (FILE *) 0x8a11008 p = <value optimized out> inspect = 0 unbuffered = 0 skipfirstline = 0 stdin_is_interactive = 1 help = 0 version = 0 saw_inspect_flag = 0 saw_unbuffered_flag = 0 cf = {cf_flags = 0} #13 0x080485b2 in main (argc=Cannot access memory at address 0x1 ) at Modules/python.c:23

          User reported that the problem was in their application.

          tabish Timothy A. Bish added a comment - User reported that the problem was in their application.

          People

            tabish Timothy A. Bish
            ssoilen Sebastien SOILEN
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: