Bug 6481 - SA 3.3.1 - Is "local.cf" read in TWICE when loading configuration files?
Summary: SA 3.3.1 - Is "local.cf" read in TWICE when loading configuration files?
Status: RESOLVED INVALID
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: spamassassin (show other bugs)
Version: 3.3.1
Hardware: PC Linux
: P4 trivial
Target Milestone: 3.3.2
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-08-09 17:32 UTC by kd6lvw+software
Modified: 2010-08-10 19:20 UTC (History)
3 users (show)



Attachment Type Modified Status Actions Submitter/CLA Status

Note You need to log in before you can comment on or make changes to this bug.
Description kd6lvw+software 2010-08-09 17:32:08 UTC
I believe that (/etc/mail/spamassassin/)"local.cf" may be read in TWICE when SpamAssassin is loading.  Why do I think that?

In my "local.cf" file, I have some additional "report" directives that append to the default report generation after the summary section.  Those rules are appearing TWICE in actual reports.  I add 4 lines, and they appear as 8 lines at the end of the report:  L1, L2, L3, L4, L1, L2, L3, and L4.  That suggests that "local.cf" is being read twice.

I run SpamAssassin as a subroutine to MIMEDefang, not as a separate process.  It is possible that MD is at fault, but I decided to open this bug here first to see if anyone can confirm this behavior.  I obtain SA via CPAN.

For anything other that reports and header additions, a second reading of the file is harmless.

Obviously, the fix is trivial.  Read "local.cf" only once and eliminate the earlier read (the latter read is so that the local file may override settings).
Comment 1 Matt Kettler 2010-08-09 22:11:11 UTC
I cannot replicate this on 3.3.1.

# spamassassin -V
SpamAssassin version 3.3.1
  running on Perl version 5.8.8

# cat /etc/mail/spamassassin/local.cf
report fooline 1
report fooline 2
report fooline3


# spamassassin </home/mkettler/Mail-SpamAssassin-3.3.1/sam
ple-spam.txt | grep fooline
fooline 1
fooline 2
fooline3


no repetition with plain spamassassin.

 spamc </home/mkettler/Mail-SpamAssassin-3.3.1/sample-spa
m.txt | grep fooline
fooline 1
fooline 2
fooline3

No repetition with spamc.

Is it possible you've got your user_prefs symlinked to your local.cf?
Comment 2 kd6lvw+software 2010-08-10 02:07:50 UTC
No symlinks (and I don't use a "user_prefs" file):

-r--r--r--  1  mailserv mail  1015 Aug  8 00:00 local.cf
            ^

I have also queried on MIMEDefang's mailing list, but no replies there yet.
Thanks for asking, but a symlink is not the cause.
(No symlinks via "ln -s" either).
Comment 3 Mark Martinec 2010-08-10 05:34:09 UTC
> For anything other that reports and header additions, a second reading
> of the file is harmless.

There are other settings too that are affected (black/whitelists,
trusted/internal/msa_networks, ...). Double reading of a .cf file
is not innocent and should not occur unintentionally. I don't think
the MIMEDefang does it, nor the SpamAssassin by itself.

> I have also queried on MIMEDefang's mailing list, but no replies there yet.

Try enabling SA debuging in your mimedefang.cf and grep the output
for 'config: read file'. See what files are really being read and when.

I don't know if there is a proper way to enable SA debugging under
mimedefang.pl, but you can add a 'debug => 1,' into the list of
arguments for a SA call in a file mimedefang.pl, e.g.:

	my $sa_args = {
		local_tests_only   => $SALocalTestsOnly,
		dont_copy_prefs    => 1,
		userprefs_filename => $config,
                debug => 1,
	};

which will let SpamAssassin write its logging to stderr. It's probably
necessary to avoid running mimedefang.pl as a server in order not to
lose stderr, but you can invoke it manually giving it a directory with
its protocol files as the only argument.
Comment 4 kd6lvw+software 2010-08-10 15:00:18 UTC
I found the problem - and a solution.  The problem was with the MIMEDefang interface, not SA proper.  Therefore, nothing need be fixed here.

When one specifies NO configuration file at all to the MD SA calls, e.g. spam_assassin_init(), MIMEDefang checks for various default files to use.  These files are in directories "etc/mail/" and "/etc/mail/spamassassin/".  The latter was the problem - in that one of the defaults it would find was in fact "/etc/mail/spamassassin/local.cf" which SA already loads on its own, so it really was being loaded TWICE.

The fix:  Specifically set the configuration (passed as "user_prefs" to SA) as "" (the empty string; not a NULL).

Since I used only spam_assassin_check(), my fix is to call it as spam_assassin_check("").

I have suggested that MIMEDefang be modified so as NOT to select default configurations from "/etc/mail/spamassassin/" in the future, since SA already has read them.
Comment 5 Kevin A. McGrail 2010-08-10 15:21:26 UTC
(In reply to comment #4)
> I found the problem - and a solution.  The problem was with the MIMEDefang
> interface, not SA proper.  Therefore, nothing need be fixed here.
> 
> When one specifies NO configuration file at all to the MD SA calls, e.g.
> spam_assassin_init(), MIMEDefang checks for various default files to use. 
> These files are in directories "etc/mail/" and "/etc/mail/spamassassin/".  The
> latter was the problem - in that one of the defaults it would find was in fact
> "/etc/mail/spamassassin/local.cf" which SA already loads on its own, so it
> really was being loaded TWICE.
> 
> The fix:  Specifically set the configuration (passed as "user_prefs" to SA) as
> "" (the empty string; not a NULL).
> 
> Since I used only spam_assassin_check(), my fix is to call it as
> spam_assassin_check("").
> 
> I have suggested that MIMEDefang be modified so as NOT to select default
> configurations from "/etc/mail/spamassassin/" in the future, since SA already
> has read them.

Hmm.  I think you are on the right track that it's not SA but I also don't think it is MD, per se.

I think the key point you have to realize is that SA isn't loading anything prior. MD is instead calling SA through an API at least by default since I actually use MD to interact with SA through spamd/spamc.

Anyway, it sounds like to me you've got multiple cf files that are being read such as sa-mimedefang.cf in /etc/mail, local.cf in /etc/mail/spamassassin and perhaps a user_prefs file somewhere perhaps under a local defang user.

I'd grep the following:

cd /etc/mail
grep report *.cf
cd /etc/mail/spamassassin
grep report *.cf

and check the home dir for whatever user is calling spamassassin.

Though to be more specific, I think you've got more than one cf file with report variables specified in one of these directories.

Especially since SA will load EVERY config (.cf) file on purpose in /etc/mail/spamassassin.

So if you have a report line in these cf files like "/etc/mail/sa-mimedefang.cf" & "/etc/mail/spamassassin.cf" as well as in local.cf, you could end up with duplicates.

Since you can fix your issue by passing "" to MD's spam_assassin_init is a "defined" config variable, you then activate this code which is why I state the examples outside of /etc/mail/spamassassin above:

       if (!defined($config)) {
            if (-r "/etc/mail/sa-mimedefang.cf") {
                $config = "/etc/mail/sa-mimedefang.cf";
            } elsif (-r "/etc/mail/spamassassin/sa-mimedefang.cf") {
                $config = "/etc/mail/spamassassin/sa-mimedefang.cf";
            } elsif (-r "/etc/mail/spamassassin/local.cf") {
                $config = "/etc/mail/spamassassin/local.cf";
            } else {
                $config = "/etc/mail/spamassassin.cf";
            }
        }
Regards,
KAM
Comment 6 kd6lvw+software 2010-08-10 18:45:59 UTC
I don't have any SA configuration files in "/etc/mail/".  Clearly, it was the double inclusion of "/etc/mail/spamassassin/local.cf" (especially since that's where I have the "report" statements that were doubly-included) which was the cause - and that was MIMEDefang's problem.  MD's documentation ("man mimedefang-filter") doesn't say that it will load any additional file if the config_file parameter (passed to SA as "user_prefs") is omitted, nor does it say for which filenames it searches.

I have told you, I have fixed the problem by passing "" in, thus disabling MD's hunt for a configuration file.
Comment 7 Karsten Bräckelmann 2010-08-10 19:20:18 UTC
Resolution INVALID, again. This is not a WONTFIX, because there's nothing to fix on SA's end. As far as SA is concerned, this just is not a bug.

Please stop changing the resolution. Even by accident. ;)  A simple reload of the page might not result in drop-down menus reflecting the current state, and thus will change it again on adding a comment. Force-reload (Shift+Ctrl-R) will do, as would open this bug report in a new browser window / tab. Thanks. :)