Robots.txt

Created
Fri, 25/08/2023 - 03:30
Updated
Fri, 25/08/2023 - 03:30

A quick note because I have to get out the door and go to work… gah.

I recently removed the restrictions on my robots.txt file.


This site has been denying access to most bots for the past year:

# Welcome to Tregeagle
######################

User-agent: archive.org_bot
Allow: /

User-agent: ia_archiver …

Tags