Bug 4978 - [review] "Minute '60' out of range" syslog errors.
Summary: [review] "Minute '60' out of range" syslog errors.
Status: RESOLVED FIXED
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Libraries (show other bugs)
Version: 3.1.3
Hardware: All All
: P3 normal
Target Milestone: 3.1.5
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard: can be commited
Keywords:
: 5361 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-07-10 17:26 UTC by Ernie Dunbar
Modified: 2007-09-04 23:43 UTC (History)
1 user (show)


Attachment Type Modified Status Actions Submitter/CLA Status
patch patch None Daryl C. W. O'Shea [HasCLA]
Alternate Patch patch None Michael Parker [HasCLA]
Patch File patch None Michael Parker [HasCLA]
error causing msg. note the Date application/octet-stream None Christoph Schulze [NoCLA]
Test case: "34 Aug 2007" Received and Date headers text/plain None Randall Jones [NoCLA]
Show Obsolete (1) Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Ernie Dunbar 2006-07-10 17:26:34 UTC 
This also goes for "Second" and "Hour" fields. 

At our site, we have a syslog scanner that looks for "unusual" messages (ie, it 
filters out the usual ones and whatever is left is unusual). This past weekend, 
I got a large number of syslog messages like this:

Jul 10 08:06:43 pop spamd[48862]: Minute '60' out of range 0..59 at /usr/local/
lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Util.pm line 429 
Jul 10 08:13:34 pop spamd[52946]: Minute '60' out of range 0..59 at /usr/local/
lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Util.pm line 429 
Jul 10 08:22:51 pop spamd[56296]: Second '60' out of range 0..59 at /usr/local/
lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Util.pm line 429 
Jul 10 08:29:23 pop spamd[56296]: Second '60' out of range 0..59 at /usr/local/
lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Util.pm line 429 

I figured this would be pretty easy to fix, and it was. I went into Util.pm and 
found that there was bounds checking on the $yyyy value in parse_rfc822_date(), 
but not $mm, $hh, $ss values. Spammers appear to be exploiting this flaw to get 
messages through spamassassin. So I just added the bounds checking. Starting at 
line 427, I added this code (minus the html preformatting tags, if you can see 
them):

<pre>
  if ($ss > 59) { 
    dbg("util: second after supported range, forcing second to 59: $date");  
    $ss = 59;
  } 
  elsif ($ss < 0) {
    dbg("util: second before supported range, forcing second to 00: $date");
    $ss = "00";
  }
  if ($mm > 59) { 
    dbg("util: minute after supported range, forcing minute to 59: $date");
    $mm = 59;
  }
  elsif ($mm < 0) {   
    dbg("util: minute before supported range, forcing minute to 00: $date");
    $mm = "00";
  }
  if ($hh > 23) { 
    dbg("util: hour after supported range, forcing hour to 23: $date"); 
    $hh = 23;
  }
  elsif ($hh < 0) {
    dbg("util: hour before supported range, forcing hour to 00: $date"); 
    $hh = "00";
  }
</pre>

This appears to have fixed the problem.
Comment 1 Daryl C. W. O'Shea 2006-07-11 02:33:05 UTC 
FWIW, I noticed a bunch of these in my logs this morning too.  All were to
accounts that reject spam though, so no samples.
Comment 2 Daryl C. W. O'Shea 2006-07-11 03:39:49 UTC 
Created attachment 3573 [details]
patch
Comment 3 Daryl C. W. O'Shea 2006-07-11 03:46:10 UTC 
Thanks Ernie.  No need for a CLA for this.

Note that this doesn't affect INVALID_DATE (which is just a header rule on the
Date header).  As far as I can see we'd rather have a slightly fudged date than
no date at all in parse_rfc822_date().

[dos@cyan trunk]$ svn ci -m "bug 4978: fudge out of range times in
parse_rfc822_date() to get usable date"
Sending        lib/Mail/SpamAssassin/Util.pm
Transmitting file data .
Committed revision 420695.
Comment 4 Justin Mason 2006-07-11 12:01:22 UTC 
+1 -- makes sense to me!
Comment 5 Theo Van Dinter 2006-07-11 13:29:16 UTC 
sure, +1
Comment 6 Daryl C. W. O'Shea 2006-07-11 21:49:34 UTC 
[dos@cyan 3.1]$ svn ci -m "bug 4978: fudge out of range times in
parse_rfc822_date() to get usable date"
Sending        lib/Mail/SpamAssassin/Util.pm
Transmitting file data .
Committed revision 421000.
Comment 7 Daniel Quinlan 2006-07-29 21:19:25 UTC 
-1 on the patch for one reason:

negative dates can't be generated from the current parsing code, so 1/2 of
the checks are slower than they need to be

if my logic is not wrong here, we should try to make this code fast since
it is called a lot for email messages
Comment 8 Daniel Quinlan 2006-07-29 21:22:22 UTC 
oops, I mean, 1/2 the checks are unnecessary, making the code slower
than it needs to be
Comment 9 Michael Parker 2006-07-29 21:23:22 UTC 
Why not just change the call to timegm_nocheck?  Adverse affects I'm not seeing?
Comment 10 Theo Van Dinter 2006-07-29 21:32:15 UTC 
moving to 3.1.5 -- patches welcome. :)
Comment 11 Michael Parker 2006-08-01 19:26:48 UTC 
Created attachment 3619 [details]
Alternate Patch

An alternative, which backs out the previous fudging and just called
timegm_nocheck, which turns off the croaks.
Comment 12 Michael Parker 2006-08-01 21:50:05 UTC 
Comment on attachment 3619 [details]
Alternate Patch

No good.  Doesn't always do the right thing.
Comment 13 Michael Parker 2006-08-01 22:47:07 UTC 
Created attachment 3620 [details]
Patch File

better, freqdiff shows no diffs.
Comment 14 Theo Van Dinter 2006-08-02 02:47:38 UTC 
+1

I was going to point out that people do have negative values in the date header:

Date: Mon, 2 Jan 2006 05:-1:29 -0300
Date: Wed, 11 Jan 2006 21:-3:28 -0400

but then I looked at the code and realized what quinlan was saying:

  if (s/ (\d?\d):(\d\d)(:(\d\d))? / /) {
    $hh = $1; $mm = $2; $ss = $4 || 0;
  }

so since \d won't match "-", we can't go < 0.  Just wanted to make this clearer. :)
Comment 15 Doc Schneider 2006-08-02 02:59:16 UTC 
+1
Comment 16 Michael Parker 2006-08-02 03:02:26 UTC 
Sending        lib/Mail/SpamAssassin/Util.pm
Transmitting file data .
Committed revision 427864.
Comment 17 Justin Mason 2006-08-02 09:16:04 UTC 
note: if we run into more of these in future, we should make a
parse_rfc822_date() test suite
Comment 18 Michael Parker 2006-08-02 17:50:55 UTC 
So, do we care about Days here?  That is one thing that the check doesn't test for.
Comment 19 Theo Van Dinter 2006-08-02 17:55:27 UTC 
(In reply to comment #18)
> So, do we care about Days here?  That is one thing that the check doesn't test
for.

Hrm.  If we're checking bounds, we may as well check all the bounds, though I'm
thinking it would be cleaner to open another ticket about "bounds checking" as
opposed to continuing this one since the initial issue has been solved.
Comment 20 Justin Mason 2006-08-02 18:03:40 UTC 
I don't think the aim of parse_rfc822_date() is to check the bounds; it's to
generate an as-correct-as-possible time_t from an RFC-822 date string.

since we don't need the day-of-week to do that, I think it's better to ignore it.
Comment 21 Theo Van Dinter 2006-08-02 18:06:09 UTC 
(In reply to comment #20)
> I don't think the aim of parse_rfc822_date() is to check the bounds; it's to
> generate an as-correct-as-possible time_t from an RFC-822 date string.
> 
> since we don't need the day-of-week to do that, I think it's better to ignore it.

I thought the comment was for day of month, which we do need, and not day of
week, which we don't.
Comment 22 Michael Parker 2006-08-02 18:13:14 UTC 
The main issue is Time::Local's croak when it gets bad input.  So, do we take
the croak and deal with it (throwing a nasty little perl error along the way) or
do we fix up the input to timegm and then call it.  Maybe the better solution is
to do bounds checking and return undef instead of trying to fix up.
Comment 23 Daryl C. W. O'Shea 2007-09-02 15:16:38 UTC 
*** Bug 5361 has been marked as a duplicate of this bug. ***
Comment 24 Justin Mason 2007-09-03 10:57:01 UTC 
*** Bug 5361 has been marked as a duplicate of this bug. ***
Comment 25 Michael Parker 2007-09-03 11:59:04 UTC 
Yes, I believe there might still be some cases that we don't handle.  I've had
them on my todo list for a really long time, just hadn't bubbled up.

If folks have example msgs please attach them to the bug so they can be tested.
Comment 26 Christoph Schulze 2007-09-04 11:28:11 UTC 
Created attachment 4115 [details]
error causing msg. note the Date

Date:	Tue, 35 Aug 2007....
Comment 27 Randall Jones 2007-09-04 18:46:35 UTC 
Created attachment 4116 [details]
Test case: "34 Aug 2007" Received and Date headers

Several similar caused 3.1.9 to generate "Day '34'" and "Day '35'" messages.  I
can scrape more samples out of our Barracuda log if you need them.
Comment 28 Theo Van Dinter 2007-09-04 19:00:29 UTC 
(In reply to comment #27)
> Test case: "34 Aug 2007" Received and Date headers
> 
> Several similar caused 3.1.9 to generate "Day '34'" and "Day '35'" messages.  I
> can scrape more samples out of our Barracuda log if you need them.

Barracuda?

Unless you can reproduce the problem with the open source SA (which I can't,
btw, none of 3.1, 3.2, or 3.3,) there's no bug here.
Comment 29 Doc Schneider 2007-09-04 19:45:55 UTC 
(In reply to comment #28)
> (In reply to comment #27)
> > Test case: "34 Aug 2007" Received and Date headers
> > 
> > Several similar caused 3.1.9 to generate "Day '34'" and "Day '35'" messages.  I
> > can scrape more samples out of our Barracuda log if you need them.
> 
> Barracuda?
> 
> Unless you can reproduce the problem with the open source SA (which I can't,
> btw, none of 3.1, 3.2, or 3.3,) there's no bug here.

My SA's version 3.1.8 is doing this. But I think it is just a attempt of the
spammers bot to get around the rules. BTW: My messages I've been seeing this
behavior with are getting caught no matter the Day setting. 

Want me to attach a sample that shows the headers that are triggering this?
Comment 30 Michael Parker 2007-09-04 20:35:29 UTC 
Better would be to patch date.t to create the failure.
Comment 31 Randall Jones 2007-09-04 23:43:11 UTC 
(In reply to comment #28)
> Barracuda?

"Spam firewall" appliance that logs most of our inbound traffic.

> Unless you can reproduce the problem with the open source SA (which I can't

Thanks for the test.  I find that our 3.1.9 is Red Hat's (with RHEL4) so I guess
I'll fall back and regroup.