IPv6 related Search Engine
Pim van Pelt
pim@bfib.ipng.nl
Tue, 30 Jan 2001 21:08:31 +0100 (CET)
Dear IPv6 folk,
The company I work for, WiseGuys Internet BV in the Netherlands, deploys
Datamining Software (webagents/spiders/search engine). I have asked their
permission to run an IPv6 related search engine at
http://www.ipng.nl/
and they have kindly agreed to license me their software.
To this end, I have started an index run on 208 websites that I know of
and feel are important to the IPv6 community. My goals are to publish
a search engine on my website in March 2001.
To this end, I'd very much like your cooperation, in two respects.
1. If you know of sites that are interresting (language irrelevant),
please let me <pim@ipng.nl> know about it. Include entrypoint and
scope of the website.
2. If you have a website, please ensure you have a robots.txt file in
your docroot, to spare your server from my relentless spiders. They will
never come anywhere that you Disallow: them to.
Guiding robots through your site is a good idea. Check out
http://www.wise-guys.nl/webagent_en.html
for strict guidelines on robots.txt and meta noindex/nofollow techniques.
Needless to say, if you are interrested in running a search engine and
are able to define a scope of websites you wish to see indexed, you can
contact WiseGuys <info@wise-guys.nl> for details.
Regards,
Pim
--
---------- - - - - -+- - - - - ----------
Pim van Pelt Email: pvanpelt@wise-guys.nl
http://www.wise-guys.nl/ GSM: +31629064049
-----------------------------------------------