在客户的网站上，有一系列教师弹出窗口位于不同的页面上。
问题是它们不需要没有索引，它们只是主页上的弹出窗口。如果直接加载链接，则它们的样式不正确。它们没有正确的html或head标签，因为它们只是通过AJAX加载。
因为它们没有头标记，所以我无法为其添加元标记。我是SEO n00b，我只知道基础知识。但我当然不认为我可以添加没有head标签的meta标签。所以我把它们添加到了robots.txt中。
我认为这将是它的结束，但我的客户正在使用SEOmoz工具栏，它说他们仍然被跟踪。我认为这根本不是问题，我认为只是工具栏没有检查robots.txt，但他会继续反复提出它。
我不知道该怎么做，所以我想澄清一下将robots.txt中的链接放入是否可行。这是我的robots.txt：
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-content/
Disallow: /wp-includes/
Disallow: /wp-content/themes/launcheducation/tutor-ajax/*?
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=90&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=90
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=359&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=359
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=362&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=362
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=365&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=365
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=367&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=367
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=369&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=369
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=371&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=371
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=373&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=373
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=897&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=897
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=900&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=900
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=902&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=902
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1003&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1003
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1017&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1017
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1020&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1020
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1023&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1023
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1031&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1031
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1123&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1123
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1297&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1297
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1300&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1300
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1305&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1305
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1669&linker=home
Disallow: /wp-content/themes/launcheducation/tutor-ajax/?id=1669
是的，我的开始只是“Disallow：/ wp-content / themes / launcheducation / tutor-ajax / *？”当插件仍未确认更改时，我将它们全部添加。 robots.txt也可以在根目录中加载：
http://launcheducation.com/robots.txt
任何帮助都会受到赞赏，即使它只是为了确认我的陈述是正确的，而且它只是不检查robots.txt的插件。如果我错了，当然，我想知道如何解决这个问题。
提前致谢。
答案 0 :(得分：1)
看起来你走在正确的轨道上。我不确定SEOMoz工具，但您应该使用的是Google网站管理员工具。在该工具中，有一项功能可让您使用Googlebot抓取任何指定的网址。这将告诉你最终是否使用robots.txt正确阻止了你的AJAX部分
如果有任何边缘情况，其中URL模式与您想要可抓取的内容匹配，则始终可以通过http标头发送NOINDEX命令，因为您无法添加元标记。 X-Robots-Tag http标头用于此目的，主要是在PDF和图像无法呈现HTML的情况下......它将适用于您的情况。
完整规格如下：https://developers.google.com/webmasters/control-crawl-index/docs/robots_meta_tag
顺便说一句，像SEOMoz这样的第三方工具很棒，但没有什么比引擎本身提供的工具更好。 ：）