Bug 4065 - FORGED_MUA_OUTLOOK / Hotmail false positives
Summary: FORGED_MUA_OUTLOOK / Hotmail false positives
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Rules (show other bugs)
Version: 3.0.2
Hardware: All All
: P5 normal
Target Milestone: 3.0.5
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
: 4181 4514 (view as bug list)
Depends on:
Blocks:
 
Reported: 2005-01-06 09:33 UTC by Tony Finch
Modified: 2005-11-13 04:01 UTC (History)
2 users (show)


Attachment Type Modified Status Actions Submitter/CLA Status
Example header extracts text/plain None Tony Finch [HasCLA]
Proposed patch based on rule above. text/plain None Chris Candreva [NoCLA]
spamassassin-3.0.4-4065-fix-FORGED_MUA_OUTLOOK.patch patch None Warren Togami [HasCLA]
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Tony Finch 2005-01-06 09:33:55 UTC 
Hotmail supports Outlook Express for access to users' mailboxes via HTTPmail,
which is actually DAV. A side-effect of this new feature is a new message-ID
format which SpamAssassin doesn't recognize. I'll attach some header extracts
with examples.
Comment 1 Tony Finch 2005-01-06 09:39:59 UTC 
Created attachment 2590 [details]
Example header extracts
Comment 2 Tony Finch 2005-01-06 09:53:01 UTC 
I think a suitable rule would be

header __OE_MSGID_3 MESSAGEID =~ /^<BAY\d+-DAV\d+[A-Z0-9]{25}\@phx\.gbl>$/
meta __FORGED_OE (__OE_MUA && !__OE_MSGID_1 && !__OE_MSGID_2 && !__OE_MSGID_3 &&
!__UNUSABLE_MSGID)
Comment 3 Chris Candreva 2005-02-01 12:38:39 UTC 
Created attachment 2637 [details]
Proposed patch based on rule above.

Here is a propsed patch, based on the rule posted above. I've applied it here,
and initial testing shows that hotmail messages that previously triggered the
rule do not do so now, while spam messages are still marked.
Comment 4 Justin Mason 2005-02-07 00:07:45 UTC 
header __OE_MSGID_3		MESSAGEID =~ /^<BAY\d+-DAV\d+[A-Z0-9]{25}\@phx\.gbl>$/
meta NEW_FORGED_OE		(__OE_MUA && !__OE_MSGID_1 && !__OE_MSGID_2 && !__OE_MSGID_3
&& !__UNUSABLE_MSGID)
Comment 5 Justin Mason 2005-02-07 00:08:11 UTC 
NEEDSMC

compare with __FORGED_OE
Comment 6 Auto-Mass-Checker 2005-02-07 14:44:36 UTC 
# [automatically generated by automc: start]
# DONEMC 5: completed request from comment 5

  0.015   0.0120   0.0215    0.359   0.44    1.00  __OE_MSGID_3_b4065_c2
  3.473   4.9982   0.0078    0.998   0.91    1.00  __FORGED_OE_b4065_c2
  0.015   0.0120   0.0215    0.359   0.44    1.00  __OE_MSGID_3_b4065_c4
  3.473   4.9982   0.0078    0.998   0.91    0.01  T_MC_NEW_FORGED_OE_b4065_c4

above freqs using data from "/home/automc/corpus/html/DETAILS.new" as of Mon Feb  7 14:44:33 2005:

__OE_MSGID_3_b4065_c2 = __OE_MSGID_3 from bug 4065 comment 2
full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__OE_MSGID_3_b4065_c2&date=20050207

__FORGED_OE_b4065_c2 = __FORGED_OE from bug 4065 comment 2
full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FORGED_OE_b4065_c2&date=20050207

__OE_MSGID_3_b4065_c4 = __OE_MSGID_3 from bug 4065 comment 4
full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__OE_MSGID_3_b4065_c4&date=20050207

T_MC_NEW_FORGED_OE_b4065_c4 = NEW_FORGED_OE from bug 4065 comment 4
full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_NEW_FORGED_OE_b4065_c4&date=20050207
# ham results used: ham-daf.log ham-jm.log ham-quinlan.log ham-rODbegbie.log
# spam results used: spam-daf.log spam-jm.log spam-quinlan.log spam-rODbegbie.log
 167577   116363    51214    0.694   0.00    0.00  (all messages)
100.000  69.4385  30.5615    0.694   0.00    0.00  (all messages as %)

bug 4065 cmt 3: ignored, lint failed

# [automatically generated by automc: end]
Comment 7 Auto-Mass-Checker 2005-02-07 14:44:43 UTC 
# [automatically generated by automc: start]
# DONEMC 5: completed request from comment 5

  0.015   0.0120   0.0215    0.359   0.44    1.00  __OE_MSGID_3_b4065_c2
  3.473   4.9982   0.0078    0.998   0.91    1.00  __FORGED_OE_b4065_c2
  0.015   0.0120   0.0215    0.359   0.44    1.00  __OE_MSGID_3_b4065_c4
  3.473   4.9982   0.0078    0.998   0.91    0.01  T_MC_NEW_FORGED_OE_b4065_c4

above freqs using data from "/home/automc/corpus/html/DETAILS.new" as of Mon Feb  7 14:44:37 2005:

__OE_MSGID_3_b4065_c2 = __OE_MSGID_3 from bug 4065 comment 2
full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__OE_MSGID_3_b4065_c2&date=20050207

__FORGED_OE_b4065_c2 = __FORGED_OE from bug 4065 comment 2
full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__FORGED_OE_b4065_c2&date=20050207

__OE_MSGID_3_b4065_c4 = __OE_MSGID_3 from bug 4065 comment 4
full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=__OE_MSGID_3_b4065_c4&date=20050207

T_MC_NEW_FORGED_OE_b4065_c4 = NEW_FORGED_OE from bug 4065 comment 4
full freqs: http://bugzilla.spamassassin.org/ruleqa?rule=T_MC_NEW_FORGED_OE_b4065_c4&date=20050207
# ham results used: ham-daf.log ham-jm.log ham-quinlan.log ham-rODbegbie.log
# spam results used: spam-daf.log spam-jm.log spam-quinlan.log spam-rODbegbie.log
 167577   116363    51214    0.694   0.00    0.00  (all messages)
100.000  69.4385  30.5615    0.694   0.00    0.00  (all messages as %)

bug 4065 cmt 3: ignored, lint failed

# [automatically generated by automc: end]
Comment 8 Justin Mason 2005-02-07 15:58:12 UTC 
applied with a minor fix (/m modifier on regexp): r151796.
Comment 9 Beat Rubischon 2005-03-09 08:20:14 UTC 
*** Bug 4181 has been marked as a duplicate of this bug. ***
Comment 10 Sidney Markowitz 2005-09-20 07:37:24 UTC 
*** Bug 4514 has been marked as a duplicate of this bug. ***
Comment 11 Warren Togami 2005-09-20 08:35:21 UTC 
Retargeting to 3.0.5.  (Currently moving across country so not able to work on
this intensively for a few days.)
Comment 12 Warren Togami 2005-10-31 03:28:14 UTC 
Created attachment 3218 [details]
spamassassin-3.0.4-4065-fix-FORGED_MUA_OUTLOOK.patch

Patch against 3.0.4.
Comment 13 Warren Togami 2005-11-07 20:17:32 UTC 
Tested patch for a week in production.

3 votes needed
Comment 14 Justin Mason 2005-11-08 01:32:25 UTC 
+1
Comment 15 Daryl C. W. O'Shea 2005-11-09 04:24:04 UTC 
+1
Comment 16 Sidney Markowitz 2005-11-13 13:01:41 UTC 
+1
Committed revision 332968.