Uploaded image for project: 'Apache RocketMQ'
  1. Apache RocketMQ
  2. ROCKETMQ-261

IndexService may block ReputMessageService

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.0.0-incubating, 4.1.0-incubating
    • Fix Version/s: 4.3.0
    • Component/s: rocketmq-store
    • Labels:
      None

      Description

      So far, ReputMessageService will do two things serially:

      1. Dispatch message to consume queue.
      2. Build message index

      IndexService uses read/write lock to ensure thread safe, build index will acquire read lock, while clean index files will hold write lock for a long time. That means clean index files will block build index, hence will block dispatch consume queue and influence real-time of consumption.

      private void deleteExpiredFile(List<IndexFile> files) {
              if (!files.isEmpty()) {
                  try {
                      this.readWriteLock.writeLock().lock();
                      for (IndexFile file : files) {
                          boolean destroyed = file.destroy(3000);
                          destroyed = destroyed && this.indexFileList.remove(file);
                          if (!destroyed) {
                              log.error("deleteExpiredFile remove failed.");
                              break;
                          }
                      }
                  } catch (Exception e) {
                      log.error("deleteExpiredFile has exception.", e);
                  } finally {
                      this.readWriteLock.writeLock().unlock();
                  }
              }
          }
      

      Action:

      Submits task 1 and 2 to different thread, to avoid mutual influence.
      Or uses lock-free algorithm in IndexService.

        Attachments

          Activity

            People

            • Assignee:
              Yukon yukon
              Reporter:
              Yukon yukon
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: