[LINK] error rates for censorware

[Michael Sims]

Peter Bowditch wrote:

> >  http://www.peacefire.org/error-rates/

> The error rates from Peacefire's study were:
> 
> CyberPatrol 1.7%
> SafeSurf 4.2%
> Bess 0.7%
> AOL 0.1%
> SafeSurfer 1%

It's pretty clear that the proper error rate should be based on the 
number of sites identified as pornographic rather than the number of 
sites in the world.  By your definition of error rate, a blacklist 
computed once and permanently fixed in stone becomes steadily more 
accurate as time goes on, simply because the size of the world wide web 
increases.  Or say that a blacklist has a grand total of one hundred 
banned sites, every single one of them wrong.  Clearly that should be a 
100% error rate, but under your definition, it's only 0.00000001% of the 
web, a miniscule error rate.  That can't be a reasonable way to look at 
it.  It's obvious that Peacefire's methodology is the correct one:

(ratio is wrongly blocked/total blocked)

Cyber Patrol     17/21   =   81% error rate
SurfWatch        42/51   =   82% error rate
Bess             7/26   =   27% error rate
AOL Parental Controls ("mature teen" setting)
                 1/5   =   20% error rate
SafeServer       10/29   =   34% error rate

In other words, when a kid at a school using Cyber Patrol to censor porn 
types in a random domain name and comes up blocked, there's about an 80% 
chance that there's no porn on that site.

Incidentally, Cyber Patrol currently blocks Peacefire's site under its 
Full Nudity and Sexual Acts categories.  And the other ten categories, 
too.  How much do blatant, intentional errors count?


--
Michael Sims - slashdot.org editor -  <michael@slashdot.org>
               Your Rights Online  - http://slashdot.org/yro
They create a desolation and call it a peace.