User-agent: * Disallow: /wp-admin/ Disallow: /jfed # Sitemap archive Sitemap: https://www.jta.org/sitemap_index.xml # Crawlers User-agent: PetalBot User-agent: MaxPoint User-agent: MaxPointCrawler User-agent: MaxPointCrawler/Nutch-1.1 User-agent: MaxPointCrawler/Nutch-1.10 User-agent: MaxPointCrawler/Nutch-1.14 User-agent: MaxPointCrawler/Nutch-1.17 Disallow: /