Uploaded image for project: 'Apache RocketMQ'
  1. Apache RocketMQ
  2. ROCKETMQ-329

Some messages was skipped by consumerOffset when rocketMQ consuming a large number of messages.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 4.1.0-incubating
    • Fix Version/s: None
    • Labels:
      None
    • Environment:
      centos 7
      one mqnamesvr, one mqbroker, with default settings.

      Description

      I started one mqnamesrv and one mqbroker process on centos7 with default settings:
      nohup bin/mqname &
      nohup bin/mqbroker -n localhost:9876 &
      and produced a large number of messages with 5 threads continuously(total 1000000 messages)´╝îfound that each consumerOffset skipped about 4500 messages when persist (as shown in the attached picture), which cause them lost while consumer consuming. The consumer lost about 16000 messages eventually.

      The speed of message production is about 5000 per second, and the speed of message consumption is 5 per second (the consumer's logic is sleep 200 millseconds and return consume success.)

      The info "the message queue not be able to consume, because it's dropped" and the warning "the pull request offset illegal" are printed in the mq's log when this condition happened. (as shown in the attached picture)

        Attachments

          Activity

            People

            • Assignee:
              yukon Xinyu Zhou
              Reporter:
              psy_core sypeng
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: