Bug 6670 - score summation sometimes produces incorrect results
Summary: score summation sometimes produces incorrect results
Status: RESOLVED INVALID
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Score Generation (show other bugs)
Version: 3.3.2
Hardware: PC Linux
: P2 normal
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-10-05 09:59 UTC by eceyesoyaz
Modified: 2011-10-18 19:28 UTC (History)
1 user (show)


Attachment Type Modified Status Actions Submitter/CLA Status
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description eceyesoyaz 2011-10-05 09:59:09 UTC 
Some sample headers with incorrectly evaluated (too low) scores.

X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on ***.***.com
X-Spam-Flag: NO
X-Spam-Status: NO, hits=5.00 required=5.00
X-Spam-Report: 
	*   10 BAYES_99 BODY: Bayes spam probability is 99 to 100%
	*      [score: 1.0000]
	*  2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
	*  0.0 TVD_RCVD_IP4 TVD_RCVD_IP4
	*  0.0 TVD_RCVD_IP TVD_RCVD_IP
	*  1.2 RCVD_NUMERIC_HELO Received: contains an IP address used for HELO
	*  1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net
	*      [Blocked - see <http://www.spamcop.net/bl.shtml?41.209.65.157>]
	*  4.5 URIBL_AB_SURBL Contains an URL listed in the AB SURBL blocklist
	*      [URIs: elit-mug.ru]
	*  1.6 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist
	*      [URIs: elit-mug.ru]
	*  1.2 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
	*      [URIs: elit-mug.ru]
	*  0.0 T_RCVD_IN_SEMBLACK RBL: Received from an IP listed in
	*      bl.spameatingmonkey.net
	*      [41.209.65.157 listed in bl.spameatingmonkey.net]
	*  0.0 T_URIBL_SEM_RED Contains a URI listed in urired.spameatingmonkey.net
	*      [URIs: elit-mug.ru]
	*  0.0 T_URIBL_SEM Contains a URI listed in uribl.spameatingmonkey.net
	*      [URIs: elit-mug.ru]
	*  1.4 RCVD_IN_BRBL_LASTEXT RBL: RCVD_IN_BRBL_LASTEXT
	*      [41.209.65.157 listed in bb.barracudacentral.org]
	*  0.0 T_SURBL_MULTI2 T_SURBL_MULTI2
	*  0.8 RDNS_NONE Delivered to internal network by a host with no rDNS
	*  0.0 T_SURBL_MULTI1 T_SURBL_MULTI1
	*  1.0 FROM_EXCESS_BASE64 From: base64 encoded unnecessarily


X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on ***.***.***
X-Spam-Flag: NO
X-Spam-Status: NO, hits=-1.50 required=5.00
X-Spam-Report: 
	*   10 BAYES_99 BODY: Bayes spam probability is 99 to 100%
	*      [score: 0.9976]
	*  0.0 FSL_HELO_NON_FQDN_1 FSL_HELO_NON_FQDN_1
	*  1.0 HK_NAME_FREE From name mentions free stuff
	*  0.0 HTML_MESSAGE BODY: HTML included in message
	*  1.6 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist
	*      [URIs: potiongrow.com]
	*  1.2 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
	*      [URIs: potiongrow.com]
	*  0.8 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
	*      [117.201.44.250 listed in dnsbl.sorbs.net]
	*  1.4 RCVD_IN_BRBL_LASTEXT RBL: RCVD_IN_BRBL_LASTEXT
	*      [117.201.44.250 listed in bb.barracudacentral.org]
	* -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high
	*      trust
	*      [117.201.44.250 listed in list.dnswl.org]
	*  0.8 RDNS_NONE Delivered to internal network by a host with no rDNS
	*  1.0 HELO_NO_DOMAIN Relay reports its domain incorrectly


X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on ***.***.***
X-Spam-Flag: NO
X-Spam-Status: NO, hits=-8.00 required=5.00
X-Spam-Report:
	*   10 BAYES_99 BODY: Bayes spam probability is 99 to 100%
	*      [score: 1.0000]
	* -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high
	*      trust
	*      [190.255.38.154 listed in list.dnswl.org]
	*  1.4 RCVD_IN_BRBL_LASTEXT RBL: RCVD_IN_BRBL_LASTEXT
	*      [190.255.38.154 listed in bb.barracudacentral.org]
	*  1.7 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist
	*      [URIs: kresloshiatsu.com]
	*  0.0 HTML_MESSAGE BODY: HTML included in message
	*  0.8 RDNS_NONE Delivered to internal network by a host with no rDNS
	*  2.5 DOS_OE_TO_MX Delivered direct to MX with OE headers


X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on ***.***.***
X-Spam-Flag: NO
X-Spam-Status: NO, hits=-1.50 required=5.00
X-Spam-Report:
	*  0.0 FSL_HELO_NON_FQDN_1 FSL_HELO_NON_FQDN_1
	*  1.0 HK_NAME_FREE From name mentions free stuff
	*  0.0 HTML_MESSAGE BODY: HTML included in message
	*  7.0 BAYES_95 BODY: Bayes spam probability is 95 to 99%
	*      [score: 0.9897]
	*  1.6 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist
	*      [URIs: cuminsided.com]
	*  1.2 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
	*      [URIs: cuminsided.com]
	*  1.3 RCVD_IN_RP_RNBL RBL: Relay in RNBL,
	*      https://senderscore.org/blacklistlookup/
	*      [46.17.153.49 listed in bl.score.senderscore.com]
	*  2.7 RCVD_IN_PSBL RBL: Received via a relay in PSBL
	*      [46.17.153.49 listed in psbl.surriel.com]
	*  1.4 RCVD_IN_BRBL_LASTEXT RBL: RCVD_IN_BRBL_LASTEXT
	*      [46.17.153.49 listed in bb.barracudacentral.org]
	*  0.0 RCVD_IN_SORBS_DUL RBL: SORBS: sent directly from dynamic IP address
	*      [46.17.153.49 listed in dnsbl.sorbs.net]
	*  0.8 RCVD_IN_SORBS_WEB RBL: SORBS: sender is an abusable web server
	* -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high
	*      trust [46.17.153.49 listed in list.dnswl.org]
	*  0.8 RDNS_NONE Delivered to internal network by a host with no rDNS
	*  1.0 HELO_NO_DOMAIN Relay reports its domain incorrectly
Comment 1 Kevin A. McGrail 2011-10-05 13:25:11 UTC 
A bug such as this would be highly unlikely.  Enough so that I would likely tell you to look at other possibilities.

For example, is a machine that ALSO runs SpamAssassin delivering the source email with headers and your system is not overwriting those headers?  

Do you have more than one machine in the mix?

In short, I can't confirm the headers for X-Spam were all produced on the same system and based on the variation between them, I would conjecture they likely aren't.

A test I often do for this is change the required to 5.01 or something similar. If I see a required=5.00 show up, I know I have an issue.

As a side note, due to the rounding in SA, there have been variances where you could get 5.0 but have it not be spam because it might be really 4.99999.  Specifically, I unified the rounding between spamc and other parts of SA somewhere in 3.3.  I need to look again and get the padding to be the same though from a different ticket.
Comment 2 Karsten Bräckelmann 2011-10-05 18:53:56 UTC 
(In reply to comment #0)
> Some sample headers with incorrectly evaluated (too low) scores.
> 
> X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on ***.***.com
> X-Spam-Flag: NO

This is a non-default header. By default, SA only includes the Flag header for spam.

> X-Spam-Status: NO, hits=5.00 required=5.00

This is a non-default header. The default Status header uses _YESNO_ template tag, not the all-caps variant. SA uses "score", rather than hits, and only shows a single digit after the decimal point. It includes a list of all tests, autolearn and version information.

> X-Spam-Report: 
>     *   10 BAYES_99 BODY: Bayes spam probability is 99 to 100%
>     *      [score: 1.0000]

The Report header by default is not added for ham, only spam. And the Level header is missing.


Did you modify all these headers yourself? Which tool actually included those headers, SA or a third-party tool?

If you did not change about every default add_header configuration option, someone else did -- possibly on a different system, and you're looking at headers added by two different systems.
Comment 3 Karsten Bräckelmann 2011-10-13 03:40:37 UTC 
eceyesoyaz, any additional info, in particular regarding the questions, you can provide?

NEEDINFO
Comment 4 Karsten Bräckelmann 2011-10-18 19:28:02 UTC 
Closing RESOLVED INVALID, as per the previous comments.

If you are sure this is not a mis-configuration, please feel free to re-open this bug, providing more detail and the information asked for.

If you suspect it might be a mis-configuration, feel free to ask for help troubleshooting the issue on the SA users mailing list.