# robots.txt # # This allows all web crawlers. # Follow the format for each directory or file that needs to be disallowed. User-agent: * Disallow: /