Affects Version/s: 4.0.0-incubating, 4.1.0-incubating
Fix Version/s: None
In RocketMQ V3.5.8, there is a readWriteLock in com.alibaba.rocketmq.store.MapedFileQueue, which guarantee thread safety. But in the new org.apache.rocketmq.store.MappedFileQueue, there is not any concurrent control mechanism.
when consumer is fetching message(no large lag), broker calls
but findMappedFileByOffset is not thread safe, as
org.apache.rocketmq.store.MappedFileQueue#deleteExpiredFile maybe running concurrently( the size of mappedFiles maybe change) , which will results in ConsumeQueue#getIndexBuffer returns null, causing
nextBeginOffset = nextOffsetCorrection(offset, consumeQueue.rollNextFile(offset));+
which will skip the whole consumeQueue file, any messages left in this ConsumeQueue will not be consumed by client.