背景(有点读)
我们创建了一个缩短的网址重定向器(使用asp.net网址路由,全部为404进行路由),其中条形码从设备扫描,然后从移动设备重定向到目标网站。我们记录所有传入的统计分析请求。自上周一晚上以来,已有大量请求和后续重定向到目标站点。通常我会说“哇喔!”但是涌入似乎过于一致,而且来自不同的IP(但是分批)和不同网址的同步请求。只有一组有效的重定向网址,但是流入只能击中那些指定的网址(即看似不是暴力攻击)。这些网址是公开提供的,但只能来自内部员工或来自已出版杂志中的代码扫描。内部没有提供任何代码。
来自请求的大多数用户代理基本上来自,显然,iPhone和大多数IP是本地的,这增加了神秘感(将显示示例记录的请求和统计数据)。 为了缓解涌入,我们首先实施了一个ip阻止规则(在代码级别上),如果3次尝试在90秒内完成,则将ip添加到阻止列表中(将该规则排除在外:oP)我们开始抓住一些。如果被阻止,我们基本上终止会话。但是这条规则可能影响了合法用户(因为它影响了内部员工......在白名单上工作但还没有完成)所以我们决定采用另一种策略,即插页式广告。
我们实施了目标网页,而不是用户需要进行互动的重定向,然后点击按钮继续。我们仍然存在针对滥用的ip阻止规则。因此,如果合法用户通过,他们只需要点击按钮继续。如果“访问者”在阻止列表中,我们实施了一个验证码例程,看看它们是否合法用于继续(不是一般的用户体验)。
我还用deny all实现了robots.txt。我甚至在登录页面中放置了一个隐藏的链接,看看是否是一个爬虫,但到目前为止,我还没有通过它跟踪任何流量。
我的技术合作伙伴认为在实际的苹果设备/应用程序/浏览器中可能会出现故障,导致它重新加载已保存的扫描网址(条形码阅读器应用程序,如RedLaser,i-nigma等等)但它只是这种情况继续发生似乎很奇怪。
因此,即使在实施着陆页后,我们仍然会获得登陆页面但未继续/点击的流量。这就是为什么我认为这是一次攻击,不是为了破坏网站,而是可能的竞争对手扭曲网站统计数据?统计数据在业务中很重要,但这是合法的统计数据。我们不想搞砸我们的客户。它也可能是内容的scapers,但是用户代理和我的陷阱无法正常工作,我不知道......实际上它是人类在点击/扫描......丢失了。
之前有没有人经历过这样的事情?
Some stats over a period of approximately 2.5 days...
Count HTTP_USER_AGENT
2924 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
506 Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
424 Mozilla/5.0 (iPod; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
401 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A523
202 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B141
180 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B144
157 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A403
138 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A405
119 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B142
119 Mozilla/5.0 (iPod; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
111 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A551
91 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B143
88 Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9A405
86 Mozilla/5.0 (iPod; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9A405
86 Mozilla/5.0 (iPod; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A523
69 Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B208
56 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A525
54 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_3 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B329
47 Mozilla/5.0 (iPod; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A403
46 Mozilla/5.0 (iPod; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B144
45 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.0 Safari/536.6
43 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B145
30 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A550
27 Mozilla/5.0 (iPod; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B141
22 Mozilla/5.0 (Linux; Android 4.1.1; SGH-I747M Build/JRO03L) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.169 Mobile Safari/537.22
21 Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; MAM3)
17 Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.2)
16 Mozilla/5.0 (iPhone; CPU iPhone OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176
13 Mozilla/5.0 (iPod; CPU iPhone OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176
13 Mozilla/5.0 (BlackBerry; U; BlackBerry 9800; en) AppleWebKit/534.8+ (KHTML, like Gecko) Version/6.0.0.600 Mobile Safari/534.8+
13 Qrafter/7.0 CFNetwork/609.1.4 Darwin/13.0.0
For these stats, I took out source IPs (for courtesy)but referenced which ones were same...just a small sample out approximate 6.5K requests
Count REMOTE_ADDR HTTP_USER_AGENT
5 A.B.C.D Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
13 A.B.C.D Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A523
2 A.B.C.D Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
6 A.B.C.D Mozilla/5.0 (iPod; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
1 B.C.D.E Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
3 B.C.D.E Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B145
10 B.C.D.E Mozilla/5.0 (iPod; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A406
15 C.D.E.F Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
31 C.D.E.F Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A405
20 C.D.E.F Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B141
2 D.E.F.G Mozilla/5.0 (iPod; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
6 D.E.F.G Mozilla/5.0 (iPod; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A403
2 D.E.F.G Mozilla/5.0 (iPod; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A523
18 E.F.G.H Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A550
49 E.F.G.H Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
15 E.F.G.H Mozilla/5.0 (iPod; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
13 E.F.I.J Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
11 E.F.I.J Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A403
11 E.F.I.J Mozilla/5.0 (iPod; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9A405
3 E.F.I.J Mozilla/5.0 (iPod; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
Sample of typical sequntial flood
URL HTTP_USER_AGENT REMOTE_ADDR LogDate
http://SITEURL/UoetZx Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:45 AM
http://SITEURL/2NedO0 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:45 AM
http://SITEURL/33hCWl Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:44 AM
http://SITEURL/e11LzG Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:44 AM
http://SITEURL/bQx5Bu Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:43 AM
http://SITEURL/BtrZ3m Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:43 AM
http://SITEURL/cxfmr1 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:43 AM
http://SITEURL/KztehQ Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:42 AM
http://SITEURL/O19sq3 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:42 AM
http://SITEURL/e6Dlwb Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:42 AM
http://SITEURL/GGQ4ZO Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:41 AM
http://SITEURL/jjr_rM Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:40 AM
http://SITEURL/yIzVel Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:40 AM
http://SITEURL/D8M0_Y Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:39 AM
http://SITEURL/-GqaX9 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:38 AM
http://SITEURL/9o0Bv8 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:37 AM
http://SITEURL/65_ce8 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:35 AM
http://SITEURL/33hCWl Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:34 AM
http://SITEURL/2NedO0 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:33 AM
http://SITEURL/UoetZx Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:33 AM
http://SITEURL/fknpPf Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:31 AM
http://SITEURL/tLEI3S Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:30 AM
http://SITEURL/MgOvvm Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:29 AM
http://SITEURL/MlJVua Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:28 AM
http://SITEURL/UcRIZj Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:27 AM
http://SITEURL/xZy-KP Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:26 AM
http://SITEURL/sXswln Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:25 AM
http://SITEURL/aQJrWx Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:24 AM
http://SITEURL/_sBrUw Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:23 AM
http://SITEURL/V7H9mK Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:22 AM
http://SITEURL/lchtkL Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:21 AM
http://SITEURL/WY7g1T Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:20 AM
http://SITEURL/bQx5Bu Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:19 AM
http://SITEURL/FznevZ Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:17 AM