Re: [Sks-devel] Request: Install an efficient robots.txt file

[Kristian Fiskerstrand]

On 06/22/2017 10:40 AM, robots.txt fan wrote:
> Kristian, you have responded to this thread, I believe you manage the first 
> one on the list. Is there a reason why only /status is blocked and not /pks?
> 
> https://sks-keyservers.net (blocks /status, but not /pks)

The real reason is that /pks didn't exist when the robots.txt file was
created, so I've [added it now], granted more for site resource
management reasons than privacy reasons.

From a privacy perspective robots.txt doesn't make sense, the data is
already public, bad actors ignore robots.txt and crawl the site just the
same; and the full data set is available and part of regular workflow
for bootstrapping own servers.

References:
[added it now]
https://git.sumptuouscapital.com/?p=sks-keyservers-pool.git;a=commit;h=b98e7522990961541165dfc23781a45a1a5e05a9

-- 
----------------------------
Kristian Fiskerstrand
Blog: https://blog.sumptuouscapital.com
Twitter: @krifisk
----------------------------
Public OpenPGP keyblock at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
----------------------------
"Whatever you do in life, surround yourself with smart people who'll
argue with you."
John Wooden

signature.asc
Description: OpenPGP digital signature