我正在编写一个 python selenium 脚本来尝试在谷歌搜索中提取 LinkedIn 个人资料的 URL 链接,但我在缩小 XPath 以仅返回谷歌上的搜索结果链接时遇到问题。
linkedin_urls = driver.find_elements_by_xpath('//div[@class="yuRUbf"]//a[@href]')
for linkedin_url in linkedin_urls:
url = linkedin_url.get_attribute("href")
print(url)
driver.get(url)
sleep(5)
linkedin_urls 给我的结果
https://uk.linkedin.com/in/roxana-andreea-popescu
https://uk.linkedin.com/in/tunjijabitta
https://www.google.com/search?source=hp&ei=bxjhX4uGC4_ykgXl9pu4Bw&q=site%3Alinkedin.com%2Fin%2F+AND+%22Software+Developer%22+AND+%22London%22&oq=site%3Alinkedin.com%2Fin%2F+AND+%22Software+Developer%22+AND+%22London%22&gs_lcp=CgZwc3ktYWIQDFDMZFjhZmCwZ2gAcAB4AIABLogBsAGSAQE0mAEAoAEBqgEHZ3dzLXdpeg&sclient=psy-ab&ved=0ahUKEwjL-dn4huDtAhUPuaQKHWX7BncQ4dUDCA0#
https://www.google.com/search?q=related:https://uk.linkedin.com/in/tunjijabitta&sa=X&ved=2ahUKEwji3qP_huDtAhWAZxUIHTyfAO4QHzABegQIBhAH
https://uk.linkedin.com/in/janomer
https://uk.linkedin.com/in/josephcoker
https://uk.linkedin.com/in/sebemin
https://uk.linkedin.com/in/vicki-marshall-b7433827
https://www.google.com/search?source=hp&ei=bxjhX4uGC4_ykgXl9pu4Bw&q=site%3Alinkedin.com%2Fin%2F+AND+%22Software+Developer%22+AND+%22London%22&oq=site%3Alinkedin.com%2Fin%2F+AND+%22Software+Developer%22+AND+%22London%22&gs_lcp=CgZwc3ktYWIQDFDMZFjhZmCwZ2gAcAB4AIABLogBsAGSAQE0mAEAoAEBqgEHZ3dzLXdpeg&sclient=psy-ab&ved=0ahUKEwjL-dn4huDtAhUPuaQKHWX7BncQ4dUDCA0#
https://www.google.com/search?q=related:https://uk.linkedin.com/in/vicki-marshall-b7433827&sa=X&ved=2ahUKEwji3qP_huDtAhWAZxUIHTyfAO4QHzAFegQIARAH
https://uk.linkedin.com/in/andreibodnar
https://www.google.com/search?q=related:https://uk.linkedin.com/in/andreibodnar&sa=X&ved=2ahUKEwji3qP_huDtAhWAZxUIHTyfAO4QHzAGegQIBxAH
https://uk.linkedin.com/in/dmrlawson
https://uk.linkedin.com/in/jack-gilbert-541a251b
https://www.google.com/search?source=hp&ei=bxjhX4uGC4_ykgXl9pu4Bw&q=site%3Alinkedin.com%2Fin%2F+AND+%22Software+Developer%22+AND+%22London%22&oq=site%3Alinkedin.com%2Fin%2F+AND+%22Software+Developer%22+AND+%22London%22&gs_lcp=CgZwc3ktYWIQDFDMZFjhZmCwZ2gAcAB4AIABLogBsAGSAQE0mAEAoAEBqgEHZ3dzLXdpeg&sclient=psy-ab&ved=0ahUKEwjL-dn4huDtAhUPuaQKHWX7BncQ4dUDCA0#
https://www.google.com/search?q=related:https://uk.linkedin.com/in/jack-gilbert-541a251b&sa=X&ved=2ahUKEwji3qP_huDtAhWAZxUIHTyfAO4QHzAIegQICxAH
https://uk.linkedin.com/in/eren-batu-999068185
我试图找到一种方法将搜索范围缩小到仅 LinkedIn 结果
答案 0 :(得分:0)
如果您只想获得 LinkedIn
结果,请在 xpath 下面使用。
使用contains()
linkedin_urls = driver.find_elements_by_xpath('//div[@class="yuRUbf"]//a[contains(@href,"https://uk.linkedin.com")]')
或starts-with()
使用
linkedin_urls = driver.find_elements_by_xpath('//div[@class="yuRUbf"]//a[starts-with(@href,"https://uk.linkedin.com")]')
答案 1 :(得分:0)
您想解析 linkedin_url
中的每个字符串,看看它是否提到了 Linkedin。
if 'linkedin' in linkedin_url:
print('linkedin')
基本上,把你想在Linkedin地址上执行的驱动代码放在if语句下。
答案 2 :(得分:0)
要将搜索限制为仅 LinkedIn 结果,您需要为 visibility_of_all_elements_located()
诱导 WebDriverWait,您可以使用以下任一 Locator Strategies:
使用 CSS_SELECTOR
:
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.yuRUbf a[href^='https://uk.linkedin.com/in']")))])
使用 XPATH
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class="yuRUbf"]//a[starts-with(@href, 'https://uk.linkedin.com/in')]")))])
注意:您必须添加以下导入:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC