robots.txt « public « front_end - crates.rs - Contrib clone of crates.rs

summary refs log tree commit diff stats

path: root/front_end/public/robots.txt

blob: cf8ba89854a620a75d7cca6898fee5342c1627a8 (plain)

User-Agent: *
Disallow: /keywords/
Disallow: /install
Disallow: /search?

User-agent: SemrushBot
Disallow: /