Uploaded image for project: 'James Server'
  1. James Server
  2. JAMES-758

InSpammerBlacklist latency seriously affects throughput

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      the following are sample mean timings from the InSpammerBlacklist matcher:

      all requests were for 127.0.0.1

      hostname - lookup count - mean timing - this lookup time

      1.0.0.127.query.bondedsender.org. - 10.0 - 71.9 - 16.0
      1.0.0.127.dnsbl.njabl.org. - 10.0 - 265.6 - 266.0
      1.0.0.127.relays.ordb.org. - 10.0 - 20095.6 - 20172.0

      As you can see they all take signifcant time, and ordb.org is painful.

      Of course "success" of the matcher is !success of the lookup, which means that while we will cache the hits they are only the failures, and the good mail will have to perform a full ns lookup everytime.

      We should think about caching the successes locally, or arranging these mailets in a separate processor and having independantly threaded processors.

      In this case (ten mails) each thread paused for 20 seconds waiting for ordb and then continued, making the whole thing pause for 20. But because I was running 10 spool threads when I increased this to 20 mails the threads paused twice, meaning that the 21st message took 40+ seconds to complete its journey through the spoolmanager.

      Not good.

      Workaround, don't use InSpammerBlacklist or set it up in a processor so that it is called by ToProcessor only on classes of mail for which the expense is justified (e.g. not on any outbound or on trusted IP's, senders or domains)

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            danny@apache.org Danny Angus
            danny@apache.org Danny Angus
            Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment