# example http://sil.org/robots.txt https://www.library.unh.edu/robots.txt User-agent: * Disallow:/