Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9
    • Fix Version/s: 0.10
    • Component/s: C++ Broker
    • Labels:
      None

      Description

      This is sometimes very hard to observe. I was lucky enough to see it three times in the space of 500 iterations yesterday, but now, with same code & scripts & system, I have no repetition after 6000 iterations.

      my tree rev number is 1084895

      Here is the backtrace – all 3 have been identical:

      #0 0x0000003f8d030265 in raise (sig=<value optimized out>)
      at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
      #1 0x0000003f8d031d10 in abort () at abort.c:88
      #2 0x0000003f8d0296e6 in __assert_fail (
      assertion=0x2b28f90400aa "completionsNeeded.get() > 0",
      file=0x2b28f903fe58 "./qpid/broker/AsyncCompletion.h", line=167,
      function=0x2b28f9040440 "void qpid::broker::AsyncCompletion::end(qpid::broker::AsyncCompletion::Callback&)") at assert.c:78
      #3 0x00002b28f8fe2935 in qpid::broker::AsyncCompletion::end (this=0x2aaab60c2e50,
      cb=<value optimized out>) at ./qpid/broker/AsyncCompletion.h:167
      #4 0x00002b28f8fdf3f3 in qpid::broker::SessionState::handleContent (
      this=0x2aaabc4157d0, frame=..., id=<value optimized out>)
      at qpid/broker/SessionState.cpp:265
      #5 0x00002b28f8fdf830 in qpid::broker::SessionState::handleIn (this=0x2aaabc4157d0,
      frame=...) at qpid/broker/SessionState.cpp:355
      #6 0x00002b28f94799dc in qpid::amqp_0_10::SessionHandler::handleIn (
      this=0x2aaab0025680, f=...) at qpid/amqp_0_10/SessionHandler.cpp:93
      #7 0x00002b28f8f2622b in operator() (this=0x2aaab38cffa0, frame=...)
      at ./qpid/framing/Handler.h:42
      #8 qpid::broker::Connection::received (this=0x2aaab38cffa0, frame=...)
      at qpid/broker/Connection.cpp:164
      #9 0x00002b28f8ef8300 in qpid::amqp_0_10::Connection::decode (this=0x2aaabc47d240,
      buffer=<value optimized out>, size=<value optimized out>)
      at qpid/amqp_0_10/Connection.cpp:58
      #10 0x00002b28f94c25e5 in qpid::sys::cyrus::CyrusSecurityLayer::decode (
      this=0x2aaabc468480, input=0x2aaab2aa97d0 "", size=118)
      at qpid/sys/cyrus/CyrusSecurityLayer.cpp:59
      #11 0x00002b28f999c7d8 in qpid::sys::RdmaIOHandler::readbuff (this=0x2aaab002d9e0,
      buff=0xffffffffffffffff) at qpid/sys/RdmaIOPlugin.cpp:218
      #12 0x00002b28f9bba75a in boost::function2<void, Rdma::AsynchIO&, Rdma::Buffer*, std::allocator<boost::function_base> >::operator() (this=0x0, a0=..., a1=0x6)

      here is my broker start script:

      -------------- start script -----------------------

      #! /bin/bash

      export LD_LIBRARY_PATH=$TRUNK/qpid/cpp/src/.libs

      QPID_SRC=$TRUNK/qpid/cpp/src
      QPIDD=$

      {QPID_SRC}/.libs/qpidd

      echo $QPIDD

      rm -rf /tmp/mick
      mkdir /tmp/mick

      $QPIDD \
      --no-module-dir \
      --load-module ${QPID_SRC}

      /.libs/rdma.so \
      --data-dir /tmp/mick/data_1 \
      --auth=yes \
      --mgmt-enable=yes \
      --port 5813 \
      --log-enable info+ \
      --log-to-file /tmp/mick/qpidd_1.log \
      --log-source yes \
      --sasl-config=$

      {QPID_SRC}

      /tests/sasl_config \
      -d

      echo "started broker from $QPIDD"
      ----------- end script ------------------------

      and here is my client iterator script;

      ---------------------- start script ---------------------
      #! /bin/bash

      rm core.* ~/.qpidd/core*

      count=0
      while [ $count -lt 10000 ]
      do
      echo "==========================================="
      echo "TEST $count"
      echo "==========================================="
      sleep 1
      core_files=`ls -l core.* ~/.qpidd/core* | wc -l`
      echo "core files: $

      {core_files}"

      if [ ${core_files}

      -gt 0 ]; then
      echo "core files found!"
      exit 1
      else
      echo "no core files found."
      fi
      ./qpid-perftest --username zig --password zig --protocol rdma --broker 20.0.40.14 --port 5813 --qt 4 --count 10
      count=$(( $count + 1 ))
      done
      ---------------------- end script -----------------------

      the sasl config directory that the broker is pointing at was created by the script cpp/src/tests/sasl_test_setup.sh

      also, you need to set "ulimit -l 131072" (at least that value) before starting the broker.

        Activity

          People

          • Assignee:
            Ken Giusti
            Reporter:
            michael j. goulish
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development