标签: robots.txt
如何禁止所有漫游器的某些网址? 像:禁止/.avi /.pdf etc..
/.avi /.pdf etc..
User-agent: * Disallow: /.mp4 Disallow: /.avi # and any other file format User-agent: googlebot Disallow: /test # Should disallow /test, /.mp4 and /.avi
这有可能吗?