Bug 5506 - URIBL quries failing
Summary: URIBL quries failing
Status: RESOLVED WORKSFORME
Alias: None
Product: Spamassassin
Classification: Unclassified
Component: Plugins (show other bugs)
Version: 3.2.0
Hardware: HP other
: P3 normal
Target Milestone: Undefined
Assignee: SpamAssassin Developer Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-06-08 15:14 UTC by Bill Landry
Modified: 2019-06-24 15:36 UTC (History)
1 user (show)


Attachment Type Modified Status Actions Submitter/CLA Status
Sample message with & without headers for URIBL testing application/octet-stream None Bill Landry [NoCLA]
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Bill Landry 2007-06-08 15:14:14 UTC 
I am finding that certain message headers cause URIBL queries to fail or not be
reported.  I will attach some sample message that when run with the header, no
URIBL hits are found.  But if the headers are removed, URIBL hits are found.

I am running SA on Fedora 7 with perl version 5.8.8.

Bill
Comment 1 Bill Landry 2007-06-08 15:25:29 UTC 
Created attachment 3981 [details]
Sample message with & without headers for URIBL testing

Attached as some sample e-mail message that fail URIBL tests when the headers
are included with the message.	However, when the headers are removed, URIBL
tests are successful.  The hyphenated messages are the sample messages with the
headers removed.

Tested using:  spamassassin -t < test.txt

Bill
Comment 2 Theo Van Dinter 2007-06-08 16:07:37 UTC 
According to my tests, SA parses the domains out just fine from both sets of
messages, and therefore queries them.  You can validate this via:

spamassassin -D uridnsbl,uri :file:test2.txt > /dev/null
versus
spamassassin -D uridnsbl,uri :file:test-2.txt > /dev/null

both result in:

dbg: uridnsbl: domains to query: statesmangamez.net

it also turns out that I get URIBL results for both, though you may see
differences depending on your environment.
Comment 3 Bill Landry 2007-06-08 16:26:47 UTC 
Fedora 7; SA 3.2.0; perl 5.8.8; Net::DNS 0.59

===========================================================
spamassassin -D uridnsbl,uri :file:test2.txt (with headers)
===========================================================
[11542] dbg: uri: html uri found,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11542] dbg: uri: cleaned html uri,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11542] dbg: uri: html domain, statesmangamez.net
[11542] dbg: uri: html uri found,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11542] dbg: uri: cleaned html uri,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11542] dbg: uri: html domain, statesmangamez.net
[11542] dbg: uri: html uri found,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11542] dbg: uri: cleaned html uri,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11542] dbg: uri: html domain, statesmangamez.net
[11542] dbg: uri: html uri found, http://statesmangamez.net/861b539/fae48.gif
[11542] dbg: uri: cleaned html uri, http://statesmangamez.net/861b539/fae48.gif
[11542] dbg: uri: html domain, statesmangamez.net
[11542] dbg: uridnsbl: domains to query: statesmangamez.net
[11542] dbg: uri: html uri found,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11542] dbg: uri: cleaned html uri,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11542] dbg: uri: html domain, statesmangamez.net
[11542] dbg: uri: html uri found,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11542] dbg: uri: cleaned html uri,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11542] dbg: uri: html domain, statesmangamez.net
[11542] dbg: uri: html uri found,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11542] dbg: uri: cleaned html uri,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11542] dbg: uri: html domain, statesmangamez.net
[11542] dbg: uri: html uri found, http://statesmangamez.net/861b539/fae48.gif
[11542] dbg: uri: cleaned html uri, http://statesmangamez.net/861b539/fae48.gif
[11542] dbg: uri: html domain, statesmangamez.net

===============================================================
spamassassin -D uridnsbl,uri :file:test-2.txt (without headers)
===============================================================
[11701] dbg: uri: parsed uri found of type parsed, http://www.w3.org/1999/xhtml
[11701] dbg: uri: cleaned parsed uri, http://www.w3.org/1999/xhtml
[11701] dbg: uri: parsed domain, w3.org
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11701] dbg: uri: cleaned parsed uri,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11701] dbg: uri: cleaned parsed uri,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11701] dbg: uri: cleaned parsed uri,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/861b539/fae48.gif
[11701] dbg: uri: cleaned parsed uri, http://statesmangamez.net/861b539/fae48.gif
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uridnsbl: domain w3.org in skip list
[11701] dbg: uridnsbl: domains to query: statesmangamez.net
[11701] dbg: uridnsbl: domain "statesmangamez.net" listed (URIBL_BLACK): 127.0.0.2
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(multi.uribl.com.:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(bl.open-whois.org.:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(uribl.swinog.ch.:statesmangamez.net)
[11701] dbg: uridnsbl: domain "statesmangamez.net" listed (URIBL_JP_SURBL):
127.0.0.80
[11701] dbg: uridnsbl: domain "statesmangamez.net" listed (URIBL_OB_SURBL):
127.0.0.80
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(multi.surbl.org.:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(white.uribl.com.:statesmangamez.net)
[11701] dbg: uridnsbl: domain "statesmangamez.net" listed (BILLS_TEST): 127.0.0.2
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(my.black.list.:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(dob.sibl.support-intelligence.net:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(sbl.spamhaus.org.:6.56.22.216)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(sbl.spamhaus.org.:114.75.94.69)
Comment 4 Bill Landry 2007-06-08 16:51:03 UTC 
Sorry, I missed the last part of the no-headers debug output:

[11701] dbg: uri: parsed uri found of type parsed, http://www.w3.org/1999/xhtml
[11701] dbg: uri: cleaned parsed uri, http://www.w3.org/1999/xhtml
[11701] dbg: uri: parsed domain, w3.org
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11701] dbg: uri: cleaned parsed uri,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11701] dbg: uri: cleaned parsed uri,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11701] dbg: uri: cleaned parsed uri,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/861b539/fae48.gif
[11701] dbg: uri: cleaned parsed uri, http://statesmangamez.net/861b539/fae48.gif
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uridnsbl: domain w3.org in skip list
[11701] dbg: uridnsbl: domains to query: statesmangamez.net
[11701] dbg: uridnsbl: domain "statesmangamez.net" listed (URIBL_BLACK): 127.0.0.2
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(multi.uribl.com.:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(bl.open-whois.org.:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(uribl.swinog.ch.:statesmangamez.net)
[11701] dbg: uridnsbl: domain "statesmangamez.net" listed (URIBL_JP_SURBL):
127.0.0.80
[11701] dbg: uridnsbl: domain "statesmangamez.net" listed (URIBL_OB_SURBL):
127.0.0.80
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(multi.surbl.org.:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(white.uribl.com.:statesmangamez.net)
[11701] dbg: uridnsbl: domain "statesmangamez.net" listed (BILLS_TEST): 127.0.0.2
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(my.black.list.:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(dob.sibl.support-intelligence.net:statesmangamez.net)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(sbl.spamhaus.org.:6.56.22.216)
[11701] dbg: uridnsbl: query for statesmangamez.net took 4 seconds to look up
(sbl.spamhaus.org.:114.75.94.69)
[11701] dbg: uri: parsed uri found of type parsed, http://www.w3.org/1999/xhtml
[11701] dbg: uri: cleaned parsed uri, http://www.w3.org/1999/xhtml
[11701] dbg: uri: parsed domain, w3.org
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11701] dbg: uri: cleaned parsed uri,
http://statesmangamez.net/tss.php?ty=12wuapuns_s36.1766S.TDP.1048
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11701] dbg: uri: cleaned parsed uri,
http://statesmangamez.net/tss.php?ty=12wuapoutexc92.1766S.TDP.1048
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11701] dbg: uri: cleaned parsed uri,
http://statesmangamez.net/tss.php?ty=12wuapexc92.1766S.TDP.1048
[11701] dbg: uri: parsed domain, statesmangamez.net
[11701] dbg: uri: parsed uri found of type parsed,
http://statesmangamez.net/861b539/fae48.gif
[11701] dbg: uri: cleaned parsed uri, http://statesmangamez.net/861b539/fae48.gif
[11701] dbg: uri: parsed domain, statesmangamez.net
Comment 5 Bill Landry 2007-06-08 17:02:25 UTC 
My issue appears to be with the Botnet plugin.  If I disable Botnet, all of the
URIBL tests run fine.  Thoughts?

I'll forward this bug report info to John Rudd, as well.

Bill
Comment 6 Mark Martinec 2007-06-12 16:36:36 UTC 
> My issue appears to be with the Botnet plugin.
> If I disable Botnet, all of the URIBL tests run fine.

See Bug 5511, which explains why URIBL test results are ignored
if some plugin (like BOTNET) takes too long for processing.
Comment 7 Henrik Krohns 2019-06-24 15:36:40 UTC 
Closing old stale bug. Probably not relevant anymore.