[LINK] The rallies???

[Michael Skeggs mike@bystander.net]

In my experience, scaling uncovers a host of issues.
I had experience some years ago with a proxy based filter system. It
required a (for then) large pair of SUN e5000 machines with full RAM
slots to provide filtered proxying to approx. 2500 dial up users.
Over time, the list of "custom" URLs added to the block list grew,
until it hit a memory limit (from memory, 1024 URLs was the limit).
The proxy did not fail gracefully when this limit was reached, it just
allowed any URLs added after 1024 on the list to be accessed with no
filtering. As the list was updated with batches of dozens of new URLs
each week, it wasn't as easy to diagnose as if it had worked with 1023
URLs then stopped with 1025.
The software supplier had not had any experience with a deployment
with so many user-added sites, and was unable to offer a solution (we
had some good sys-admins who figured it out, and their band-aid fix
was pretty smart too).
Eventually, the solution was to reverse engineer the blocking list (by
running unfiltered logs from a public squid proxy against the filtered
proxy, and collecting the banned URLs) and use an OSS proxy with this
list redirecting to "This site is blocked" page.
Lessons learned:
- the latency of filtering was *acceptable* for dial-up applications
- the filter folk do not have expertise in networking, they have
expertise in collecting porn URLs - so their software isn't as good as
standard software
- the number of sites with potentially objectionable content on the
web is large and ever growing
- the filter folk's list of banned sites has collateral damage (e.g.
breast cancer screening sites, reproductive health) and as the lists
are trade secrets, it is impossible to know up front what the
collateral damage is.
- the filter software folk have limited experience of large scale
deployment, despite claims to the contrary.
- blocking non-http sites is not practical.

Regards,
Michael Skeggs


2008/12/17 Danny Yee <danny at anatomy.usyd.edu.au>:
> Glen hits a key issue on the head: scaling.
>
> I doubt any of the companies pushing filtering products and claiming
> marvels for them have any experience with deployments larger than a
> school, or perhaps a large company.  ISP backbones are on an entirely
> different scale, both in topological complexity - where do you put the
> filtering devices? - and bandwidth.
>
> Some of the other issues also scale poorly.  If there is overblocking
> by a school or company, that won't cause terrible harm to a business
> that gets blocked accidentally - it will only affect a small number
> of customers, and those customers won't necessarily expect to be
> able to do business in a study/work setting anyway.  An accidental
> Australian-Internet-wide block, on the other hand, could destroy an
> online business serving Australian markets.
>
> So a government mandated national blacklist needs much better
> accountability and stronger review mechanisms than organisation-level
> filtering systems.  But the ACMA blacklist is actually less transparent
> and less accountable than many of the commercial filtering blacklists!
>
> Any latency or bandwidth degradation from filtering is also easier
> to cater for at a school or company level.  A small organisation
> can know what its critical traffic is and have that bypass the
> filter, for example, or simply know that they have no critical
> communications.  A backbone-level filter is going to have to know
> which of its connections are, or could potentially be, used in
> performance-critical operations.  Are hospitals going to be able to
> apply for exemptions from filtering on the grounds that they might be
> carrying out telesurgery?  How would such an exemption system work?
> Deploying any kind of robust quality-of-service system for IP is
> still an unsolved problem!
>
> Danny.
>
> Glen Turner wrote:
>>  From a design point of view it's a useless number.
>        .
>        .
>
> _______________________________________________
> Link mailing list
> Link at mailman.anu.edu.au
> http://mailman.anu.edu.au/mailman/listinfo/link
>