lynx-dev
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: lynx-dev Web robots


From: Philip Webb
Subject: Re: lynx-dev Web robots
Date: Mon, 7 Sep 1998 17:11:02 -0400 (EDT)

980907 Jonathan Lawson wrote: 
> does anyone know about web robots and how to write/implement them?

my personal & ever-growing file of useful URLs tells me:

Agents                      www.opensesame.com/agents/
Agents: I/Net intelligence  www.cs.umbc.edu/agents/
  FAQ & listserv            www.ee.mcgill.ca/~belmarc/agent_faq.html
User-agent string           www.idrc.ca/cgi-bin/bc.cgi
  ditto                     sailor.lib.md.us/cgi-bin/find-agents
Robot searchers :  info.webcrawler.com/mak/projects/robots/robots.html
                     -/-/-/-/norobots.html
 
> 'They came forth from unholy darknesses ...
>  and were driven back by the rage of Angels'
 
robot spammers, maybe?

-- 
========================,,============================================
SUPPORT     ___________//___,  Philip Webb : address@hidden
ELECTRIC   /] [] [] [] [] []|  Centre for Urban & Community Studies
TRANSIT    `-O----------O---'  University of Toronto

reply via email to

[Prev in Thread] Current Thread [Next in Thread]