[Remops] EMP Filters and Me

[drsnoid]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 16 Mar 2009 10:40:35 +0000, you wrote:
>
> On Sat, Mar 14, 2009 at 10:36:12AM -0400, drsnoid wrote:
> >
> > Well, there is something amiss then.  If I compare those messages which
> > according to the mail2news log have either been accepted and posted or
> > reported as existing duplicates with what actually appears in
> > alt.testing.testing, there are very many missing.  This is evident on any
> > news server I read from, and interestingly news.mixmin.net has the *fewest*
> > - - one would expect it to show the greatest according to the mail2news log.
>
> Indeed, and if some are going missing I'd like to understand why.  Do
> you have any sample Message-ID's of ones missing on Mixmin but posted
> elsewhere?

By the time it was all said and done there were two which appeared on other
servers but not mixmin, both from Frell:

Subject: huge 3 test frell post
Message-ID: <fbb1d49fce9d870f4d3eb062fd3a3697 at msgid.frell.theremailer.net>
and
Subject: huge 3 test frell banana
Message-ID: <0a40ea08f1ed06ea26b16a5b38b754f9 at msgid.frell.theremailer.net>

Of course there were many others which did not appear on any server which I
believe should have, at least there was never any problem in the past.

> > > Ah, I wasn't aware that posting-host was a part of the equation.  I've also
> > wondered if age were one of an EMP filter's criteria (does "substantially
> > identical" expire after n days?)
>
> No.  There are primarily three types of EMP filter:-
> PHL	-	NNTP-Posting-Host / Lines
> MD5	-	Message Payload
> FSL	-	From / Subject / Lines
> All of these are created by MD5 hashing the criteria.  The hash is then
> stored in a list along with a counter.  Each time a hash collision
> occurs the associated counter is incremented until a threshold is
> crossed and further messages are rejected.  Over time the counters
> decrement until they reach zero and the hash is deleted.

Thanks for the information.

The day before yesterday I devised a way of inserting my own special brand
of random garbage into the message bodies and made another run, which seems
to have alleviated the problem.  Some new results should be along shortly!

But it all makes me wonder what changed on such a system-wide basis.  Never
use to be a problem, then suddenly ZAP, no matter which mail2news, exit
remailer, or news server my messages seemed unwelcome.

Best wishes,
drsnoid

-----BEGIN PGP SIGNATURE-----
Version: N/A

iQA/AwUBSb5rvpc5FWApPF04EQLAawCg2T7FgcZ1rYKYdm4Fu8KG7Lxav8gAn1oG
Z1mGf885j4Vpx8JVtlRZYDPi
=3PjV
-----END PGP SIGNATURE-----