Created attachment 5111 [details]
sample mail message

(In reply to comment #0)
> Today I was investigating a false positive for a mail from our
> Japanese research colleagues at KEK.jp. It turns out that the following
> perfectly valid From line caused a collection of 6.1 score points
> solely because there is no (optional) space between a display name
> and the address:

Granted it's valid, but it's apparently very common in spam generated by sloppy tools and not common in mail from well-written MUAs. Could you get the X-Mailer from the headers so we know what generated that From: line? That header was not in the spample you attached.

Please see if this message (and any other similar FPs) can be added to a masscheck ham corpus so that the generated scores can better reflect this.

I will see about tuning the rules; any change I make will probably be rather focused, either to having an encoded display name with no space, or using that specific mailer.

I also note the lack of a space after the To: - whatever mailer your colleague is using needs some cleanup. :)

> Granted it's valid, but it's apparently very common in spam generated by
> sloppy tools and not common in mail from well-written MUAs. Could you get
> the X-Mailer from the headers so we know what generated that From: line?
> That header was not in the sample you attached.

The attached header is pretty much complete, I just replaced the username
with xxx and yyy, simplified Subject, and dropped local Received entries.
Unfortunately there was no X-Mailer or User-Agent header field.

> I also note the lack of a space after the To: - whatever mailer your
> colleague is using needs some cleanup. :)

Indeed there was no (optional) space after the To: . The message is a
user registration procedure confirmation, apparently generated by some
automation scripting or some HR management software. Body is all in
ascii plain text, doesn't reveal much of its creation procedure,
just states their Users Office contact address, a password and a
few instructions.

> Please see if this message (and any other similar FPs) can be added to a
> masscheck ham corpus so that the generated scores can better reflect this.

Not sure if I can reveal more of the message beyond what was attached.

> I will see about tuning the rules; any change I make will probably be rather
> focused, either to having an encoded display name with no space, or using
> that specific mailer.

I'm not complaining about each of these three rules scores individually,
but their cumulative action seems excessive for a simple unusual formatting.

Changing spamassassin for a single false positive clearly caused by poorly written software seems weird.  But I think we could still use some work improving automated handling of overlapping rules.

(In reply to comment #3)
> > Please see if this message (and any other similar FPs) can be added to a
> > masscheck ham corpus so that the generated scores can better reflect this.
> 
> Not sure if I can reveal more of the message beyond what was attached.

What you posted should be sufficient, just ensure it hits all of the FP rules you are concerned about.

I notice that the original message hit FROM_MISSP_URI but there's no URI in the sample you provided. For completeness there should be one; if you don't want to leak the real URI then replace it with something innocuous like a link to google or your home page or something.

Created attachment 5112 [details]
replaced a sample mail message

> What you posted should be sufficient, just ensure it hits
> all of the FP rules you are concerned about.
> I notice that the original message hit FROM_MISSP_URI but
> there's no URI in the sample you provided. For completeness
> there should be one;

Done - attached.

The rule FROM_MISSP_URI has been disabled in r1441795 and the rule FROM_MISSP_EH_MATCH is not triggered by this sample email any more.