我正在从LinkedIn销售导航器中刮取个人资料。所有配置文件都在列表项Sample StackBlitz
中当我尝试获取所有列表项时,只会找到前10个。我在做什么错了?
sel = Selector(text=driver.page_source)
names = sel.xpath('//li[@class="pv5 pr6 search-results__result-item"]')
len(names)
10
>>> print(names)
[<Selector xpath='//li[@class="pv5 pr6 search-results__result-item"]'
data='<li class="pv5 pr6 search-results__resul'>, <Selector
xpath='//li[@class="pv5 pr6 search-results__result-item"]' data='<li
class="pv5 pr6 search-results__resul'>, <Selector xpath='//li[@class="pv5
pr6
search-results__result-item"]' data='<li class="pv5 pr6 search-
results__resul'>, <Selector xpath='//li[@class="pv5 pr6 search-
results__result-item"]' data='<li class="pv5 pr6 search-results__resul'>,
<Selector xpath='//li[@class="pv5 pr6 search-results__result-item"]'
data='<li class="pv5 pr6 search-results__resul'>, <Selector
xpath='//li[@class="pv5 pr6 search-results__result-item"]' data='<li
class="pv5 pr6 search-results__resul'>, <Selector xpath='//li[@class="pv5
pr6
search-results__result-item"]' data='<li class="pv5 pr6 search-
results__resul'>, <Selector xpath='//li[@class="pv5 pr6 search-
results__result-item"]' data='<li class="pv5 pr6 search-results__resul'>,
<Selector xpath='//li[@class="pv5 pr6 search-results__result-item"]'
data='<li class="pv5 pr6 search-results__resul'>, <Selector
xpath='//li[@class="pv5 pr6 search-results__result-item"]' data='<li
class="pv5 pr6 search-results__resul'>]
答案 0 :(得分:0)
我找到了解决方案。我需要硒来向下滚动到页面。我使用了这段代码:
scheight = .1
while scheight < 9.9:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight/%s);" %
scheight)
scheight += .01
找到了该帖子的答案:Efficient method to scroll though pages using Selenium