# Make changes for all web spiders
User-agent: *
Disallow: /cp-public
Disallow: /lex
Disallow: /rte-snippets
Disallow: /templates
Disallow: /Test