Re: New question about interaction between spamass-milter and clamav.

[EASY address@hidden]

On Tue, 2009-07-07 at 14:15 -0400, Steven W. Orr wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 07/07/09 13:21, quoth EASY address@hidden:
> > On Tue, 2009-07-07 at 13:06 -0400, Steven W. Orr wrote: On 07/07/09 13:00, 
> > quoth EASY address@hidden:
> >>>> On Tue, 2009-07-07 at 12:42 -0400, Steven W. Orr wrote: I need some 
> >>>> help with whether I'm doing something wrong.
> >>>> 
> >>>> Here's my setup:
> >>>> 
> >>>> I have sendmail-8.14.3 spamass-milter-0.3.1-13 (set to reject) 
> >>>> spamassassin-3.2.5
> >>>> 
> >>>> All was good in the world, but slowly the spam that got through was 
> >>>> creeping up.
> >>>> 
> >>>> I found that if I added clamav-milter that a lot of the stuff would 
> >>>> get caught. I came to this list a while back to ask about the pros 
> >>>> and cons of whether the clamav-milter should go before or after 
> >>>> spamass-milter. I decided on doing clamav-milter first. Both milters 
> >>>> were set to reject.
> >>>> 
> >>>> Then the false negs started climbing again and I heard about all the 
> >>>> clamav addons that were available via scamp from
> >>>> 
> >>>> https://sourceforge.net/projects/scamp/
> >>>> 
> >>>> That made a huge difference and life was good again.
> >>>> 
> >>>> Again, things were escalating and I decided that the real problem is 
> >>>> that every time clamav rejects something, spamassassin doesn't learn 
> >>>> from it. The SA AWL and the bayes tables never get informed except 
> >>>> for the false negs that get through that get fed to sa-learn --spam. 
> >>>> I thought that it would make more sense for SA to run the clam test 
> >>>> itself. I looked around and sure enough I found clamav.pm  which is a
> >>>>  plugin for SA.
> >>>> 
> >>>> So where I am now is that I have eliminated the clamav-milter, and 
> >>>> I'm running a clamd so that clamscam works, and I installed the 
> >>>> clamav plugin to SA. The sendmail incantation I'm using is this 
> >>>> standard one:
> >>>> 
> >>>> INPUT_MAIL_FILTER(`spamassassin', 
> >>>> `S=local:/var/run/spamass-milter/spamass-milter.sock, F=, 
> >>>> T=C:15m;S:4m;R:4m;E:10m')dnl
> >>>> 
> >>>> and the params for running spamass-milter are
> >>>> 
> >>>> -m -u steveo -r 5 -d misc -i 192.168.0.1/24 -i 127.0.0.1/24
> >>>> 
> >>>> I can't believe you've read this far. But if you got here then I can 
> >>>> now ask my question:
> >>>> 
> >>>> Why is it that I am now seeing  my server asking for retries? I am 
> >>>> getting rejects like I expect to get (and like I got before). For 
> >>>> example,
> >>>> 
> >>>> Jul  5 05:17:17 saturn spamass-milter[3478]: queueid=n659HG7c007407 
> >>>> Jul  5 05:17:18 saturn sendmail[7407]: n659HG7c007407: 
> >>>> from=<address@hidden>, size=2174, class=0, nrcpts=1, 
> >>>> msgid=<address@hidden>, proto=ESMTP, 
> >>>> daemon=MTA, relay=sombody_good [good.guy.we.like] Jul  5 05:17:18 
> >>>> saturn sendmail[7407]: n659HG7c007407: Milter add: header: 
> >>>> X-Virus-Scanned: ClamAV 0.94.2/9538/Fri Jul  3 10:27:11 2009 on 
> >>>> myserver.syslang.net Jul  5 05:17:18 saturn sendmail[7407]: 
> >>>> n659HG7c007407: Milter add: header: X-Virus-Status: Infected with 
> >>>> Sanesecurity.Spam.10285.UNOFFICIAL Jul  5 05:17:19 saturn 
> >>>> sendmail[7407]: n659HG7c007407: Milter: data, reject=554 5.7.1 virus 
> >>>> Sanesecurity.Spam.10285.UNOFFICIAL detected by ClamAV - 
> >>>> http://www.clamav.net Jul  5 05:17:19 saturn sendmail[7407]: 
> >>>> n659HG7c007407: to=<address@hidden>, delay=00:00:01, 
> >>>> pri=32174, stat=virus Sanesecurity.Spam.10285.UNOFFICIAL detected by 
> >>>> ClamAV - http://www.clamav.net
> >>>> 
> >>>> 
> >>>> But now I'm also seeing retries when clamav is deciding that the 
> >>>> status is unknown. Somehow, spamass-milter is returning a 4.5.1
> >>>> 
> >>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: 
> >>>> from=<address@hidden>, size=3879, class=0, nrcpts=1, 
> >>>> msgid=<address@hidden>,
> >>>>  proto=ESMTP, daemon=MTA, relay=201-94-178-5.jau.flash.tv.br 
> >>>> [201.94.178.5]
> >>>> 
> >>>> 
> >>>> 
> >>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: Milter add: 
> >>>> header: X-Virus-Scanned: ClamAV 0.94.2/9538/Fri Jul  3 10:27:11 2009 
> >>>> on myserver.syslang.net
> >>>> 
> >>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: Milter add: 
> >>>> header: X-Virus-Status: Unknown
> >>>> 
> >>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: Milter: data,
> >>>>  reject=451 4.3.2 Please try again later
> >>>> 
> >>>> Jul  5 04:55:56 saturn sendmail[19195]: n658tqMf019195: 
> >>>> to=<address@hidden>, delay=00:00:01, pri=33879, stat=Please try
> >>>>  again later
> >>>> 
> >>>> I'm not sure that I have a complaint. It's just that I did not have 
> >>>> any notion that spamass-milter had the ability to do anything but 
> >>>> either accept or reject with a 500 series code and I wanted to 
> >>>> understand what was going on. I figured that SA only returns an 
> >>>> assessment, but it's spamass-milter that does the actual rejecting.
> >>>> 
> >>>> Can someone explain this? (And again, thanks for reading.)
> >>>> 
> >>>> 
> >>>>> 
> > 
> >>>> It makes sense to virus scan first and drop. In an ideal world you 
> >>>> should be taking out as much 'obvious' rubbish as early as you can in
> >>>>  the SMTP stage. Only give Spamassassin anything that other anti UCE 
> >>>> methods cannot latch on to. Spamassassin takes some tweaking. I won't
> >>>>  use the Bayes feature at all - but that is a personal thing. You
> >>>> have to add rules that meet your requirements and see the kind of
> >>>> trends you have. Some examples of the false positives put up at : 
> >>>> http://pastebin.com/ may get you some suggestions.
> > Ok, but you're actually not on the same subject as what I'm asking: How is 
> > it possible for spamass-milter to returns a 400 series retry request? I 
> > have it set to either reject if the score is greater than 5 or accept.
> > 
> > I am NOT asking whether I should do virus checking first. I'm also not 
> > asking for how to fine tune SA. I just don't see how these retries are 
> > possible given the way it's configured.
> > 
> > 
> > My apologies. I did not spot the 400 question. My inclination is this will 
> > be specific to the MTA. It is the MTA that does the 4xx (defer) or 5xx 
> > (reject). The milter simple says 'yes' or 'no'. The only reason I could see
> >  a 451 and that would be if the milter was unavailable (misconfigured or 
> > down). I've seen this with Postfix when the milter is unreachable. (Had 
> > some issues with permissions on the unix socket). My guess is sendmail has 
> > a similar mechanism.
> 
> > If you are sure the milter is up and running and you can feed a test telnet
> >  message all the way through, then somewhere your MTA is set to give a 
> > temporary fail rather than permanent.
> 
> Excellent! We're back on track. The milter is up and running. The spamd is
> running and clamd is running. What I see in the log files are 100% consistent:
> 
> If SA calls the CLAMAV plugin and it's clean then no score is added because of
>  the plugin. But then I'm left with two possibilities. The first is that the
> milter rejects because the clamav plugin said that it is a virus. The second
> is like this:
> 
> Jul  5 09:37:03 saturn sendmail[17349]: n65Db1c4017349: Milter add: header:
> X-Virus-Status: Unknown
> Jul  5 09:37:03 saturn sendmail[17349]: n65Db1c4017349: Milter: data,
> reject=451 4.3.2 Please try again later
> 
> So someone is saying unknown. (That's the plugin) That unknown is then being
> reported back by spamass-milter with a 451 and then from there it's somehow
> turned into a "4.3.2 Please try again later".
> 
> It's not making sense to me, if for no other reason that the milter is
> supposed to only either accept or reject based on whether the score is greater
> than the threshold (or not).
> 
> Does this help?
> 
> - --
> Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
> happened but none stranger than this. Does your driver's license say Organ ..0
> Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
> individuals! What if this weren't a hypothetical question?
> steveo at syslang.net
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.10 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAkpTkNIACgkQRIVy4fC+NyQpMQCfZ2n9QIO19mDofgMN3bjDjBJc
> dakAnifGtP5BiKjsy+VHFEOeUH6OE0JZ
> =vyEW
> -----END PGP SIGNATURE-----

Simple question for ten points. If you disable the clamav in SA, and
feed a message through does it work or are you getting the deferral?

I'm not 100% at the order of play here, but my guess would be:
clamav -> dns/network based tests -> spam scanning. Can you confirm
without the clamav running the rest works flawlessly?