废弃角度列表数据

Question

import requests
from bs4 import BeautifulSoup

url= requests.get('https://angel.co/companies').text
soup= BeautifulSoup(url, 'lxml')

for div in soup.find_all("div", class_="name"):
    print(div.text)

我想打印公司名单的名称，但它什么都不打印。

Answer 1

As @Shahin mentioned in his comment, you will likely have better luck using Selenium to get the content of the webpage. Specify the URL you would like to load and I would implement a scroll feature to the bottom of the page (I am pretty familiar with angel.co and their website contains a large number of companies, especially depending on your search terms). I will leave it to you to launch the driver for Selenium:

from bs4 import BeautifulSoup
from selenium import webdriver

link = 'Specify your link here'
driver.get(link)

'Enter code here to support scrolling through the webpage'

soup = BeautifulSoup(driver.page_source, 'html.parser')
driver.quit()

Answer 2

import requests
from selenium import webdriver
from bs4 import BeautifulSoup
import time

driver=webdriver.Chrome()
driver.get('https://angel.co/companies?locations[]=1637-Dubai,+AE')
time.sleep(10)
soup= BeautifulSoup(driver.page_source, 'lxml')
name_data = []
for item in soup.select('.name'):
    name=item.text.strip()
    name_data.append(name)
    print(name)


driver.quit()

工作正常