Uploaded image for project: 'Apache RocketMQ'
  1. Apache RocketMQ
  2. ROCKETMQ-332

MappedFileQueue is not thread safe, which will cause message loss.

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.0.0-incubating, 4.1.0-incubating
    • Fix Version/s: None
    • Component/s: rocketmq-store
    • Labels:
      None

      Description

      In RocketMQ V3.5.8, there is a readWriteLock in com.alibaba.rocketmq.store.MapedFileQueue, which guarantee thread safety. But in the new org.apache.rocketmq.store.MappedFileQueue, there is not any concurrent control mechanism.

      when consumer is fetching message(no large lag), broker calls
      org.apache.rocketmq.broker.processor.PullMessageProcessor#processRequest ==>
      org.apache.rocketmq.store.DefaultMessageStore#getMessage ==>
      org.apache.rocketmq.store.ConsumeQueue#getIndexBuffer ==>
      org.apache.rocketmq.store.MappedFileQueue#findMappedFileByOffset

      but findMappedFileByOffset is not thread safe, as
      org.apache.rocketmq.store.MappedFileQueue#deleteExpiredFile maybe running concurrently( the size of mappedFiles maybe change) , which will results in ConsumeQueue#getIndexBuffer returns null, causing
      nextBeginOffset = nextOffsetCorrection(offset, consumeQueue.rollNextFile(offset));+
      which will skip the whole consumeQueue file, any messages left in this ConsumeQueue will not be consumed by client.

        Attachments

          Activity

            People

            • Assignee:
              Yukon yukon
              Reporter:
              Jas0n918 Jas0n918
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: