Bug 5776

Summary: concatenation errors in Plugin/Check.pm
Product: Spamassassin Reporter: Adam Stephens <adam.stephens>
Component: LibrariesAssignee: SpamAssassin Developer Mailing List <dev>
Status: RESOLVED FIXED    
Severity: normal CC: alex.kiernan, bugzilla.spamassassin.org, micah
Priority: P5    
Version: 3.2.4   
Target Milestone: 3.3.1   
Hardware: Sun   
OS: Solaris   
Whiteboard:

Description Adam Stephens 2008-01-09 02:28:45 UTC
Upgrading to 3.2.4 caused thousands of concatenation errors in the logs:

Jan  9 10:13:53 localhost spamd[16335]: Use of uninitialized value in
concatenation (.) or string at
/usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/Plugin/Check.pm line 1026.
Jan  9 10:13:53 localhost last message repeated 329 times

I'm seeing about 7000 lines of this per second:
# grep "Jan  9 10:13:54" /var/adm/spamd | grep -c line\ 1026
7220

Reverting to 3.2.3 fixes the problem.
Comment 1 Justin Mason 2008-01-09 02:43:46 UTC
do you use allow_user_rules?

have you additional rules defined in /etc/mail/spamassassin ?

does "spamassassin --lint" pass?
Comment 2 Justin Mason 2008-01-09 02:46:47 UTC
oh -- for reference, line 1026 is marked by ### below:


sub hash_line_for_rule {
  my ($self, $pms, $rulename) = @_;
  return "\n".'#line 1 "'.                           ###
        $pms->{conf}->{source_file}->{$rulename}.
        ', rule '.$rulename.',"';
}
Comment 3 Adam Stephens 2008-01-09 07:39:02 UTC
> do you use allow_user_rules?
>
>

No.

> have you additional rules defined in /etc/mail/spamassassin ?
>
>
Yes.

Ah! It seems to be caused by my shortcircuit.cf, modified from the one on the SA
wiki
(http://wiki.apache.org/spamassassin/ShortcircuitingRuleset)

If I comment out the following rule:

# simple, non-network-based whitelists, locally-generated messages,
# messages via a trusted relay chain, simple, non-network based blacklists
meta SC_HAM
(USER_IN_WHITELIST||USER_IN_DEF_WHITELIST||USER_IN_ALL_SPAM_TO||NO_RELAYS||USER_IN_BLACKLIST_TO||USER_IN_BLACKLIST)
priority SC_HAM -1000
shortcircuit SC_HAM ham
score SC_HAM -20

the error stops.

> does "spamassassin --lint" pass?
>
>
Yes
Comment 4 Micah 2008-09-11 22:07:31 UTC
I had the same problem when I upgraded from 3.2.3 to 3.2.5. Downgrading to 3.2.4 did not solve it, but returning to 3.2.3 did. 

I also narrowed it down to the ShortCircuit plugin, when disabled it stopped filling my log files.

Removing the lines suggested in the previous entry also made it go away, although these seem like they were useful short-circuits.
Comment 5 Dale Blount 2008-10-31 12:32:52 UTC
This is happening to me also on a few boxes, but I have one where it is not running the same spamassassin packages.  

I can't find allow_user_rules anywhere in the config directory, so I'm assuming I'm running whatever is default.

spamassassin --lint passes.

quite a few rules additional rules are defined as well.

Disabling the shortcut plugin in v320.pre fixes it.

I'm just adding short circuiting now, so I have no idea if this was a problem with previous builds or a regression.

I can provide additional debugging if required.
Comment 6 Dale Blount 2008-11-03 07:58:27 UTC
Forgot to mention, this is happening to me on Intel/Linux, so it's not a Solaris or Sun specific bug.
Comment 7 Dale Blount 2008-11-05 12:36:04 UTC
I just tested this on another box with the same linux version and packages but couldn't reproduce it.  I then took one of the spamd boxes that had the issue out of RR DNS and did some testing with it.  I couldn't reproduce it with any sample messages I had (ones that SC'd, and ones that didn't), so it's either a load issue or a formatting issue on incoming email.  The first and second messages scanned after putting it back into RR DNS both triggered the error.  It also appears to have crippled the chances for new children to start, it seems to get hung at 2 children.

Unlike the original poster, my spamd is pretty much unusable when it's spewing that many log entries and I have to disable the plugin again.
Comment 8 Michael Parker 2008-11-05 12:52:45 UTC
> Unlike the original poster, my spamd is pretty much unusable when it's spewing
> that many log entries and I have to disable the plugin again.

Are you saying that you're disabling the Check plugin?  Don't do that.  Without that plugin your SA install will not scan messages at all and should be producing loud warnings.
Comment 9 Dale Blount 2008-11-05 12:53:39 UTC
(In reply to comment #8)
> > Unlike the original poster, my spamd is pretty much unusable when it's spewing
> > that many log entries and I have to disable the plugin again.
> 
> Are you saying that you're disabling the Check plugin?  Don't do that.  Without
> that plugin your SA install will not scan messages at all and should be
> producing loud warnings.
> 


No, I disabled the shortcircuit plugin.
Comment 10 Dale Blount 2009-03-11 13:39:08 UTC
I've noticed the default 

shortcircuit USER_IN_WHITELIST       on
shortcircuit USER_IN_DEF_WHITELIST   on
shortcircuit USER_IN_ALL_SPAM_TO     on
shortcircuit SUBJECT_IN_WHITELIST    on
shortcircuit USER_IN_BLACKLIST       on
shortcircuit USER_IN_BLACKLIST_TO    on
shortcircuit SUBJECT_IN_BLACKLIST    on

doesn't seem to cause the errors but adding

shortcircuit SC_HAM ham

breaks it.  Should it be   "shortcircuit SC_HAM  on  ham" maybe?

Comment 11 Dale Blount 2009-05-27 09:00:41 UTC
Any chance this will get fixed in 3.3.0?  I can't find any commits that appear to touch this.  I've noticed a severe lack of others having this issue - is there something we can do to debug why we're having the problem and others aren't?

This is perl, v5.10.0 built for i686-linux-thread-multi

Intel hardware, Linux 2.6.26 i686.

What changed here between 3.2.3 and 3.2.4?
Comment 12 Mark Martinec 2009-10-30 10:39:31 UTC
> Any chance this will get fixed in 3.3.0?  I can't find any commits that appear
> to touch this.  I've noticed a severe lack of others having this issue - is
> there something we can do to debug why we're having the problem and others
> aren't?
> This is perl, v5.10.0 built for i686-linux-thread-multi
> Intel hardware, Linux 2.6.26 i686.
> What changed here between 3.2.3 and 3.2.4?

I can't reproduce this with current 3.3 SVN code when running a command-line
spamassassin, even when using the mentioned rules. No idea.
Comment 13 Justin Mason 2010-01-27 02:20:00 UTC
moving most remaining 3.3.0 bugs to 3.3.1 milestone
Comment 14 Justin Mason 2010-01-27 03:14:58 UTC
these were not supposed to go to security!  moving back out of that component, but now probably to the wrong components :( , but better than nothing.
Comment 15 Justin Mason 2010-01-27 03:15:24 UTC
these were not supposed to go to security!  moving back out of that component, but now probably to the wrong components :( , but better than nothing.
Comment 16 Justin Mason 2010-01-27 03:16:05 UTC
reassigning, too
Comment 17 Micah 2010-03-05 04:34:17 UTC
I've updated to a newer version of Debian (Lenny) and am using 3.3.0 and I re-enabled the Short Circuit plugin and am no longer seeing this problem.
Comment 18 Mark Martinec 2010-03-05 10:17:50 UTC
> I've updated to a newer version of Debian (Lenny) and am using 3.3.0 and I
> re-enabled the Short Circuit plugin and am no longer seeing this problem.

Thanks! I'm closing this as resolved. Please re-open if the problem reappears.