[LINK] making the index transparent (was Re: Police raid home of Wikileaks.de domain owner over censorship lists

[Adrian Chadd]

On Sat, Mar 28, 2009, Marghanita da Cruz wrote:

> <snip>
> 
> ISPs host multiple virtual websites on single hosts - my understanding is that
> they would all share the same IP address.
> 
> Thus blocking the IP of a prohibited site will result in the blocking of sites
> belonging to other customers of that ISP. Hence, I guess the need for takedowns.

The point is to use multiple "filters" to narrow down what needs to be intercepted
and scanned. Scanning -everything- is a very difficult problem to get right
given the constraints (eg, how the SP network is built. :)

Having an IP hash; tying into intercepting DNS lookup requests/replies; these
allow a filter mechanism to selectively intercept traffic per-IP rather than
having to potentially scan 10s of gigabits of traffic.

I've done it as a proof of concept on a small scale here out of sheer boredom. :)



Adrian