Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-10225

Utility to rate limit writes in case async indexing is delayed

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.52.0, 1.22.17
    • indexing
    • None

    Description

      An application might writes too much stuff too quickly to the repository. In the extreme case, it might even write so much that the application might need to be stopped.

      To avoid this, rate-limiting the writes would be good. For this, we can add a utility class with a method rateLimitWrites(), which can be called in order to limit the writes, in case the async indexes are lagging behind badly.

      The method should return immediately if all async indexes are up-to-date (updated in the last 30 seconds).

      If indexing lanes are lagging behind, however, the method should wait (using Thread.sleep) for at most 1 minute. If the method is called more than once per minute, it should sleep for at most the time that passed until the last call; that is, an application that is calling it a lot will be paused for up to 50%. This assumes indexes will be able to catch up in this situation.

      Attachments

        Activity

          People

            thomasm Thomas Mueller
            thomasm Thomas Mueller
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: