Re: Interesting spamming tool: a robot to grab e-mail addresses
- Date: Mon, 30 Mar 1998 16:27:12 +0100
> > Bull's Eye Gold is the PREMIER email address collection tool.
> > ...
> > ... All you need is your web browser and this program.
> > Our software utilizes the latest in search technology called
> > "spidering". By simply feeding the spider program a starting
> > website it will collect for hours. The spider will go from website
> > to targeted website providing you with thousands upon thousands of
> > fresh TARGETED email addresses.
>
> Hm, I suspect that this software does not care about robots.txt.
> I can't think of an easy way to stop this sort of free enterprise
> at the doorstep of a webserver..
Here's an idea (presuming it ignores robots.txt files, and forges its
User-Agent to look like a popular browser) Use wpoison (or similar) to
construct an infinitely deep area of your web tree. List this area in
your robots.txt file.
If you see more than (x) hits to this area in a certain time from a
certain IP address then set up access control measures to block that
IP address from accessing your server. Perhaps return a message
telling them why this is happened, and how to have their access
re-enabled. You could do all of this automatically, so the admin
wouldn't have to do anything about it.
<Flight of fantasy>
You could even set up something similar to the RBL - where a method could
exist for maintaining a global list of "bad" IP addresses, who would then
be barred from accessing web servers that used that list.
</Flight of fantasy>
Cheers,
Simon.
