Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
4.0.0-incubating, 4.1.0-incubating
-
None
-
None
Description
In RocketMQ V3.5.8, there is a readWriteLock in com.alibaba.rocketmq.store.MapedFileQueue, which guarantee thread safety. But in the new org.apache.rocketmq.store.MappedFileQueue, there is not any concurrent control mechanism.
when consumer is fetching message(no large lag), broker calls
org.apache.rocketmq.broker.processor.PullMessageProcessor#processRequest ==>
org.apache.rocketmq.store.DefaultMessageStore#getMessage ==>
org.apache.rocketmq.store.ConsumeQueue#getIndexBuffer ==>
org.apache.rocketmq.store.MappedFileQueue#findMappedFileByOffset
but findMappedFileByOffset is not thread safe, as
org.apache.rocketmq.store.MappedFileQueue#deleteExpiredFile maybe running concurrently( the size of mappedFiles maybe change) , which will results in ConsumeQueue#getIndexBuffer returns null, causing
nextBeginOffset = nextOffsetCorrection(offset, consumeQueue.rollNextFile(offset));+
which will skip the whole consumeQueue file, any messages left in this ConsumeQueue will not be consumed by client.