Question

我使用XPath //*[@href]来抓取网页上的链接。但是，我注意到除了页面上的实际链接外，Selenium还在抓取javascript:void(0)。

为什么会这样？

如果你在http://google.com上运行此测试，你会发现45个链接 - 其中3个实际上不是链接，而是javascript：void（0）。

如何解决？

更新：http://google.com上的预期输出（截至撰写本文时）如下：

1 = https://www.google.com/images/branding/product/ico/googleg_lodp.ico
2 = https://www.google.com/
3 = https://www.google.com/setprefs?suggon=2&prev=https://www.google.com/?gws_rd%3Dssl&sig=0_OQUUDCX_hZxBr1qNxxxxxxxxxxxEH_4%3D
4 = https://mail.google.com/mail/?tab=wm
5 = https://www.google.com/imghp?hl=en&tab=wi&ei=3NfSWL2xxxxxxxBg&ved=0EKouCBgoAQ
6 = https://www.google.com/intl/en/options/
7 = https://myaccount.google.com/?utm_source=OGB
8 = https://www.google.com/webhp?tab=ww&ei=3NfSWL2DKpxxxxxxxg&ved=0EKkuCAIoAQ
9 = https://maps.google.com/maps?hl=en&tab=wl
10 = https://www.youtube.com/
11 = https://play.google.com/?hl=en&tab=w8
12 = https://news.google.com/nwshp?hl=en&tab=wn&ei=3NfSWL2xxxxxxxxxxxBg&ved=0EKkuCAYoBQ
13 = https://mail.google.com/mail/?tab=wm
14 = https://drive.google.com/?tab=wo
15 = https://www.google.com/calendar?tab=wc
16 = https://plus.google.com/?gpsrc=ogpy0&tab=wX
17 = https://translate.google.com/?hl=en&tab=wT
18 = https://photos.google.com/?tab=wq&pageId=none
19 = https://www.google.com/intl/en/options/
20 = http://www.google.com/shopping?hl=en&tab=wf&ei=3NxxxxxxxxxTYBg&ved=0EKkuCA0oDA
21 = https://wallet.google.com/?tab=wa
22 = https://www.google.com/finance?tab=we
23 = https://docs.google.com/document/?usp=docs_alc
24 = https://books.google.com/bkshp?hl=en&tab=wp&ei=3NfSWL2xxxxxxxxxxxBg&ved=0EKkuCBEoEA
25 = https://www.blogger.com/?tab=wj
26 = https://www.google.com/contacts/?hl=en&tab=wC
27 = https://hangouts.google.com/
28 = https://keep.google.com/
29 = https://www.google.com/intl/en/options/
30 = https://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=https://www.google.com/%3Fgws_rd%3Dssl
----------------------- (removed)
32 = https://www.google.com/url?q=https://www.google.com/intl/en_us/homepage/search/sp-firefox.html%3Futm_source%3Dgoogle.com%26utm_medium%3Dpushdown%26utm_content%3Dswitch%26utm_campaign%3Dffdse&source=hpp&id=190xx319&ct=7&usg=AFxxxxxxxxbZR_QouKfSxxxxxxxuQ&cot=2
33 = https://www.google.com/webhp?hl=en&sa=X&ved=0ahUKxxxxxy8-rSAxxxxxxxx8QPAgD
34 = https://support.google.com/websearch/answer/186645?hl=en
35 = https://www.google.com/intl/en/policies/privacy/?fg=1
36 = https://www.google.com/intl/en/policies/terms/?fg=1
37 = https://www.google.com/preferences?hl=en
38 = https://www.google.com/preferences?hl=en&fg=1
39 = https://www.google.com/advanced_search?hl=en&fg=1
40 = https://www.google.com/history/optout?hl=en&fg=1
41 = https://support.google.com/websearch/?p=ws_results_help&hl=en&fg=1
------------------ (removed)
43 = https://www.google.com/intl/en/ads/?fg=1
44 = https://www.google.com/services/?fg=1
45 = https://www.google.com/intl/en/about.html?fg=1

Answer 1

如果你真的需要用＆＃34; href＆＃34;属性除了这两个链接，那么你可以使用下一个xPath：

//*[@href][not(contains(@href,'javascript:void'))]

Answer 2

如果您想获得包含URL作为参考的锚点，可以在XPath表达式下方使用

"//a[starts-with(@href, 'http')]"

这应涵盖URLs和http架构的https

XPath // * [@ href]不仅刮取链接

2 个答案: