Infrastructure
  1. Infrastructure
  2. INFRA-5331

asf mailer rejects almost half of my mails

    Details

      Description

      I'm having troubles sending to ASF mailing lists lately.
      All my mails are plaintext (MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii) and do not contain links. But still get rejected every 2nd time. This happens only since a few weeks.
      1. rejected-mail.log
        2 kB
        Mark Struberg
      2. Delivery report 1.dat
        0.5 kB
        Jacques Le Roux
      3. Delivery report 2.dat
        0.5 kB
        Jacques Le Roux
      4. Delivery report 3.dat
        0.5 kB
        Jacques Le Roux
      5. Delivery report 4.dat
        0.5 kB
        Jacques Le Roux
      6. Force builds on buildbot 1.eml
        2 kB
        Jacques Le Roux
      7. Force builds on buildbot 2.eml
        2 kB
        Jacques Le Roux
      8. Force builds on buildbot 3.eml
        3 kB
        Jacques Le Roux
      9. Re_ Hot to obtain product localized content through HTML services_ 4.eml
        3 kB
        Jacques Le Roux

        Activity

        Hide
        Sebb added a comment -
        It would help infra if you provided some examples of the failure messages (as attachments).
        Show
        Sebb added a comment - It would help infra if you provided some examples of the failure messages (as attachments).
        Hide
        Mark Struberg added a comment -
        Hi Sebb, see the attached file - txs!
        Show
        Mark Struberg added a comment - Hi Sebb, see the attached file - txs!
        Hide
        Sebb added a comment -
        Are there any other rejects with a different set of entries in the spam score line?

        The one you have attached says:

        Remote host said: 552 spam score (6.0) exceeded threshold (FREEMAIL_FORGED_REPLYTO,FSL_FREEMAIL_1,FSL_FREEMAIL_2,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL ) [BODY]

        I'm guessing here, but the Reply-To uses @yahoo.de, whereas the mail is actually sent by yahoo.com.
        Maybe this causes FREEMAIL_FORGED_REPLYTO ?
        [I don't know what the other settings mean]
        For the mails that do get through, I suggest you check the headers to see if they are being sent by yahoo.de or yahoo.com.
        Maybe yahoo changed the way it sends mails recently.
        Show
        Sebb added a comment - Are there any other rejects with a different set of entries in the spam score line? The one you have attached says: Remote host said: 552 spam score (6.0) exceeded threshold (FREEMAIL_FORGED_REPLYTO,FSL_FREEMAIL_1,FSL_FREEMAIL_2,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL ) [BODY] I'm guessing here, but the Reply-To uses @yahoo.de, whereas the mail is actually sent by yahoo.com. Maybe this causes FREEMAIL_FORGED_REPLYTO ? [I don't know what the other settings mean] For the mails that do get through, I suggest you check the headers to see if they are being sent by yahoo.de or yahoo.com. Maybe yahoo changed the way it sends mails recently.
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> Note that another Y! user also reported problems recently (infra-p) and with the same set of spamassassin rules (except also HTML_* there).
        Show
        #asfinfra IRC Bot added a comment - <danielsh> Note that another Y! user also reported problems recently (infra-p) and with the same set of spamassassin rules (except also HTML_* there).
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> And that user is US based, I think only .com was involved.
        Show
        #asfinfra IRC Bot added a comment - <danielsh> And that user is US based, I think only .com was involved.
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> We've received multiple reports of this problem (all from yahoo users) so we'll zero out the new spamassassin rules. That should fix the problem for mails sent after the upcoming midnight UTC
        Show
        #asfinfra IRC Bot added a comment - <danielsh> We've received multiple reports of this problem (all from yahoo users) so we'll zero out the new spamassassin rules. That should fix the problem for mails sent after the upcoming midnight UTC
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> Thanks to Joe for the fix.
        Show
        #asfinfra IRC Bot added a comment - <danielsh> Thanks to Joe for the fix.
        Hide
        Mark Struberg added a comment -
        Thanks a lot for the quick solution!
        Show
        Mark Struberg added a comment - Thanks a lot for the quick solution!
        Hide
        Jacques Le Roux added a comment -
        Is there a new rule in spamassassin which prohibits to have urls in mail content? I got 2 messages rejected recently and I don't see any other reasons, please find them attached, thanks
        Show
        Jacques Le Roux added a comment - Is there a new rule in spamassassin which prohibits to have urls in mail content? I got 2 messages rejected recently and I don't see any other reasons, please find them attached, thanks
        Hide
        Jacques Le Roux added a comment -
        OK I got a copy of the 4th message to get to the OFBiz user ML using the corresponding Nabble forum. So I guess my email client (still Outlook Express on XP :D) is part of the pb. I will try with the OFBiz dev ML message. Maybe I will finally retry to move to another email client (after failing in 2005 with Thunberbird due to Outlook Express filter rules not being "backportedable")
        Show
        Jacques Le Roux added a comment - OK I got a copy of the 4th message to get to the OFBiz user ML using the corresponding Nabble forum. So I guess my email client (still Outlook Express on XP :D) is part of the pb. I will try with the OFBiz dev ML message. Maybe I will finally retry to move to another email client (after failing in 2005 with Thunberbird due to Outlook Express filter rules not being "backportedable")
        Hide
        Jacques Le Roux added a comment -
        OK it worked, so it seems Outlook Express (OE) is now almost banished? I can't change/remove the reply-type in OE (see http://mail-archives.apache.org/mod_mbox/spamassassin-commits/200806.mbox/%3C20080616122447.11621.4371@eos.apache.org%3E), or at least I did not find how.

        From

        <dev@ofbiz.apache.org>: host mx1.us.apache.org[140.211.11.136] said: 552 spam
            score (7.3) exceeded threshold
            (RCVD_IN_DNSWL_NONE,SPF_NEUTRAL,STOX_REPLY_TYPE,STOX_REPLY_TYPE_WITHOUT_QUOTES,TO_NO_BRKTS_MSFT,URI_HEX
            (in reply to end of DATA command)

        and

        <dev@ofbiz.apache.org>: host mx1.eu.apache.org[192.87.106.230] said: 552 spam
            score (6.0) exceeded threshold
            (RCVD_IN_DNSWL_NONE,SPF_NEUTRAL,STOX_REPLY_TYPE,STOX_REPLY_TYPE_WITHOUT_QUOTES,TO_NO_BRKTS_MSFT
            (in reply to end of DATA command)


        I read at http://spamassassin.apache.org/tests_3_3_x.html (default scores maybe not the ones used at the ASF...)

        rule = score
        RCVD_IN_DNSWL_NONE = neglectable
        SPF_NEUTRAL = 0.8 max
        STOX_REPLY_TYPE = 1.9 max
        STOX_REPLY_TYPE_WITHOUT_QUOTES = ?
        TO_NO_BRKTS_MSFT = ?
        URI_HEX = 2.8

        0.8 + 1.9 + 2.8 = 5.5
        0.8 + 1.9 = 2.7

        it seems it's mostly an email client issue ( if I look at the 2nd I find: 6 - 2.7 = 3.3 for
        STOX_REPLY_TYPE_WITHOUT_QUOTES = ?
        TO_NO_BRKTS_MSFT = ?
        which I guess are both related to OE. So could we not find an agreement on how to not banish OE as email client?
        Show
        Jacques Le Roux added a comment - OK it worked, so it seems Outlook Express (OE) is now almost banished? I can't change/remove the reply-type in OE (see http://mail-archives.apache.org/mod_mbox/spamassassin-commits/200806.mbox/%3C20080616122447.11621.4371@eos.apache.org%3E), or at least I did not find how. From < dev@ofbiz.apache.org >: host mx1.us.apache.org[140.211.11.136] said: 552 spam     score (7.3) exceeded threshold     (RCVD_IN_DNSWL_NONE,SPF_NEUTRAL,STOX_REPLY_TYPE,STOX_REPLY_TYPE_WITHOUT_QUOTES,TO_NO_BRKTS_MSFT,URI_HEX     (in reply to end of DATA command) and < dev@ofbiz.apache.org >: host mx1.eu.apache.org[192.87.106.230] said: 552 spam     score (6.0) exceeded threshold     (RCVD_IN_DNSWL_NONE,SPF_NEUTRAL,STOX_REPLY_TYPE,STOX_REPLY_TYPE_WITHOUT_QUOTES,TO_NO_BRKTS_MSFT     (in reply to end of DATA command) I read at http://spamassassin.apache.org/tests_3_3_x.html (default scores maybe not the ones used at the ASF...) rule = score RCVD_IN_DNSWL_NONE = neglectable SPF_NEUTRAL = 0.8 max STOX_REPLY_TYPE = 1.9 max STOX_REPLY_TYPE_WITHOUT_QUOTES = ? TO_NO_BRKTS_MSFT = ? URI_HEX = 2.8 0.8 + 1.9 + 2.8 = 5.5 0.8 + 1.9 = 2.7 it seems it's mostly an email client issue ( if I look at the 2nd I find: 6 - 2.7 = 3.3 for STOX_REPLY_TYPE_WITHOUT_QUOTES = ? TO_NO_BRKTS_MSFT = ? which I guess are both related to OE. So could we not find an agreement on how to not banish OE as email client?
        Hide
        Sebb added a comment -
        I just tried sending an e-mail to my GMail account with OE (XP), and the headers are as follows:


        MIME-Version: 1.0
        Content-Type: text/plain; charset="iso-8859-1"
        Content-Transfer-Encoding: 7bit
        X-Priority: 3
        X-MSMail-Priority: Normal
        X-Mailer: Microsoft Outlook Express 6.etc.
        X-MimeOLE: Produced By Microsoft MimeOLE V6.etc
        X-MDF-HostID: dd

        No sign of the "reply-type=original" header which is apparently what is triggering your rejection.
        Show
        Sebb added a comment - I just tried sending an e-mail to my GMail account with OE (XP), and the headers are as follows: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.etc. X-MimeOLE: Produced By Microsoft MimeOLE V6.etc X-MDF-HostID: dd No sign of the "reply-type=original" header which is apparently what is triggering your rejection.
        Hide
        Jacques Le Roux added a comment -
        Thanks Sebastian, I will digg in a bit to see what I can do then
        Show
        Jacques Le Roux added a comment - Thanks Sebastian, I will digg in a bit to see what I can do then
        Hide
        Jacques Le Roux added a comment -
        OK, I tried UTF-8 charset, still reply-type. Then tried a bunch of other changes, finally keeping original iso-8859-1 and using quoted-printable seems to do it, thanks to my POP3 doing X-MIME-Autoconverted I can still read French accents I sent to myself. I will see what others receive,hopefully all POP3 server are like mine smart enough...

        Content-Type: text/plain; charset=iso-8859-1
        X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.2.0
        X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on mx-transit7.nfrance.com
        Content-Transfer-Encoding: 8bit
        X-MIME-Autoconverted: from quoted-printable to 8bit by albane.nfrance.com id q97CgKwn046669

        I wanted to try that for a while, but not sure it's the right solution either reading http://mailformat.dan.info/config/oex.html:

        The Plain Text Settings box determines some formatting for plain text messages. (I hope you chose to use plain text instead of HTML!) You can choose MIME or UUEncode encoding; this determines how attachments are coded. UUEncode is really archaic now, so MIME is the better choice. However, within the MIME section, you should pick "None" for the encoding choice. If you do this, you'll technically be breaking the standard if you use any characters outside the 7-bit US-ASCII range (e.g., if you include an accented letter), but choosing "Quoted Printable" (which would encode such characters and bring the message into compliance with the standards) unfortunately causes Outlook Express to violate another standard: that of line length. The "Automatically wrap text" item gets grayed out if "Quoted Printable" is used, and hard linebreaks are inserted only at the end of paragraphs, causing all sorts of display problems for other mail programs and Web-based mailing list archives. I used to think that this line-length problem happened in the plain-text versions of Outlook Express messages sent in HTML form, and went away when you picked plain text instead (yet another reason to prefer plain text over HTML), but it turns out that it's actually the "quoted printable" setting (which is by default on for HTML messages but off for plain text messages) that causes the problem.

        So maybe like suggested Gavin to me, the best way would be to turn to Outlook 2010 that I have already installed, thanks to the MS license, no hurry...
        Show
        Jacques Le Roux added a comment - OK, I tried UTF-8 charset, still reply-type. Then tried a bunch of other changes, finally keeping original iso-8859-1 and using quoted-printable seems to do it, thanks to my POP3 doing X-MIME-Autoconverted I can still read French accents I sent to myself. I will see what others receive,hopefully all POP3 server are like mine smart enough... Content-Type: text/plain; charset=iso-8859-1 X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=disabled version=3.2.0 X-Spam-Checker-Version: SpamAssassin 3.2.0 (2007-05-01) on mx-transit7.nfrance.com Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by albane.nfrance.com id q97CgKwn046669 I wanted to try that for a while, but not sure it's the right solution either reading http://mailformat.dan.info/config/oex.html: The Plain Text Settings box determines some formatting for plain text messages. (I hope you chose to use plain text instead of HTML!) You can choose MIME or UUEncode encoding; this determines how attachments are coded. UUEncode is really archaic now, so MIME is the better choice. However, within the MIME section, you should pick "None" for the encoding choice. If you do this, you'll technically be breaking the standard if you use any characters outside the 7-bit US-ASCII range (e.g., if you include an accented letter), but choosing "Quoted Printable" (which would encode such characters and bring the message into compliance with the standards) unfortunately causes Outlook Express to violate another standard: that of line length. The "Automatically wrap text" item gets grayed out if "Quoted Printable" is used, and hard linebreaks are inserted only at the end of paragraphs, causing all sorts of display problems for other mail programs and Web-based mailing list archives. I used to think that this line-length problem happened in the plain-text versions of Outlook Express messages sent in HTML form, and went away when you picked plain text instead (yet another reason to prefer plain text over HTML), but it turns out that it's actually the "quoted printable" setting (which is by default on for HTML messages but off for plain text messages) that causes the problem. So maybe like suggested Gavin to me, the best way would be to turn to Outlook 2010 that I have already installed, thanks to the MS license, no hurry...
        Hide
        #asfinfra IRC Bot added a comment -
        <danielsh> assumed solved, if not reopen with a specific problem description
        Show
        #asfinfra IRC Bot added a comment - <danielsh> assumed solved, if not reopen with a specific problem description

          People

          • Assignee:
            Unassigned
            Reporter:
            Mark Struberg
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development