Subject: Re:  New: bayesdbm.t, test 37 takes ages

On Sun, Feb 15, 2004 at 02:38:56PM -0800, bugzilla-daemon@bugzilla.spamassassin.org wrote:

> The call seems to be: 
>  
> /usr/bin/perl -T -w ../sa-learn -C log/test_rules_copy \ 
>   --siteconfigpath log/localrules.tmp -p log/test_default.cf  \ 
>   --ham data/whitelists
> 

It's learning 25 messages in the t/data/whitelists directory.  Takes
about 22 seconds on my machine.  I don't think it's anything code
wise, just exercising the sa-learn code a bit.

Michael

It's actually taking much longer than I thought. I think I've got to enable 
debugging for the sa-learn tests to see what its doing there... 
 
mss@otherland ~/projects/spamassassin/trunk.clean $ time make test \ 
  TEST_FILES=t/bayesdbm.t 
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" 
"test_harness(0, 'blib/lib', 'blib/arch')" t/bayesdbm.t 
t/bayesdbm....ok 
All tests successful. 
Files=1, Tests=43, 75 wallclock secs (15.90 cusr +  0.53 csys = 16.43 CPU) 
 
real    1m15.969s 
user    0m16.055s 
sys     0m0.570s 
 

Subject: Re:  bayesdbm.t, test 37 takes ages

I wonder if this has anything to do with the sleep for 1 second that
Justin posted about the other day.

http://xrl.us/bhef

Michael

Maybe. Taking bug for now, let's see what I can do about it... 

Subject: Re:  bayesdbm.t, test 37 takes ages

On Mon, Feb 16, 2004 at 03:41:52PM -0800, bugzilla-daemon@bugzilla.spamassassin.org wrote:
> 
> I wonder if this has anything to do with the sleep for 1 second that
> Justin posted about the other day.
> 
> http://xrl.us/bhef
> 

Indeed.  I was using 4.1.25, upgrading to 4.2.52 seems to make the
delay go away.  I went from 1m39.295s to 0m21.568s to run all of the
tests.

Question: Should we just call this a Berkeley DB bug (possibly this
one from their change log: Fix a bug where contention in the buffer
pool could cause the buffer allocation algorithm to unnecessarily
sleep waiting for buffers to be freed. [#7572])? or should we figure
out if we're just tickling it somehow and correct that.

I do know that I can do one run that will display the delay, and
another run that will not have the delay.  Not sure of why that is,
maybe the db is already in RAM or something like that.

Michael

Subject: Re:  bayesdbm.t, test 37 takes ages

On Fri, Feb 20, 2004 at 07:22:13AM -0800, bugzilla-daemon@bugzilla.spamassassin.org wrote:
> Indeed.  I was using 4.1.25, upgrading to 4.2.52 seems to make the
> delay go away.  I went from 1m39.295s to 0m21.568s to run all of the
> tests.

Heh.

> Question: Should we just call this a Berkeley DB bug (possibly this
> one from their change log: Fix a bug where contention in the buffer
> pool could cause the buffer allocation algorithm to unnecessarily
> sleep waiting for buffers to be freed. [#7572])? or should we figure
> out if we're just tickling it somehow and correct that.

IMO, we should see if there's a way to work around their bug since
4.1.25 is a standard for certain OS distros out there (RHEL [AE]S 3
for instance).  If there isn't, or it isn't easy, then calling it a bug
in their lib is fine and we can just note it in the FAQ.

I wouldn't spend too much time on this since it's a known bug in the lib.

Subject: Re:  bayesdbm.t, test 37 takes ages

> IMO, we should see if there's a way to work around their bug since
> 4.1.25 is a standard for certain OS distros out there (RHEL [AE]S 3
> for instance).  If there isn't, or it isn't easy, then calling it a bug
> in their lib is fine and we can just note it in the FAQ.

We can and should ask the Berkeley DB folks if there's a way to work
around it.

Also note in INSTALL, I'd say.
 
> I wouldn't spend too much time on this since it's a known bug in the lib.

Agreed.

Just a data point, someone on the subversion list noticed a similar 1 sec pause
in Berkeley DB:
http://www.contactor.se/~dast/svn/archive-2004-02/0805.shtml
Might be memory related.
I'll watch and see if they come up with any answers.

adding note to INSTALL and FAQ