Question

我在这里使用此代码在图片网站上查找图片的 URL：

import requests
from bs4 import BeautifulSoup
import urllib.request

inp = input("search:")

url= "https://unsplash.com/s/photos/"+ inp

response = requests.get(url)

soup = BeautifulSoup(response.content, "html.parser")

images= soup.find_all("img", attrs={"class": "_2UpQX" })
print(len(images)

虽然它找到了页面上的第一张图片，但它并没有找到所有的甚至接近这个（比如 6/500 或类似的东西）

这段代码有什么问题？是不是因为页面本身没有加载所有内容？

感谢您的帮助。

Answer 1

数据是通过 JavaScript 以 JSON 格式动态加载的。要从各个页面加载图像 URL，您可以使用以下示例：

import json
import requests


search = "tree"

api_url = "https://unsplash.com/napi/search/photos"
query = {"query": search, "per_page": "20", "page": "1", "xp": ""}

for page in range(1, 3):  # <--- increase number of pages here
    query["page"] = page
    data = requests.get(api_url, params=query).json()
    for result in data["results"]:
        print(result["urls"]["full"])
    print("-" * 80)

打印：

https://images.unsplash.com/photo-1556108936-019b2ccd9c11?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8MzgxfHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1602280493900-4570f57b9efc?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8MzgyfHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1614691211055-dee3dc6eb80d?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8MzgzfHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1593052583071-d5c8bb438720?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzg0fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1579964208918-0e6752566887?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzg1fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1586660491914-63f40d885b97?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzg2fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1588249888625-8ac37a9f264d?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzg3fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1615146064145-14d6e59650fc?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzg4fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1610193450857-1cb58cb64b8c?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzg5fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1587691812967-d0dc0b0684c6?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8MzkwfHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1607808137325-4e0c8b04b23b?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8MzkxfHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1602592818543-90a1b47dc967?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8MzkyfHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1591969739980-0d99dcaa71b3?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8MzkzfHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1597517339515-ec57f0de7561?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzk0fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1614616475159-9eb631ad0873?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzk1fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1609700931477-11b704af81c8?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzk2fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1611488533271-e953994e0877?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzk3fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1616636584787-207781fc78ef?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzk4fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1541500233866-71164d920e0b?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Mzk5fHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1553537093-d3d970825195?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8NDAwfHx0cmVlfGVufDB8fHx8MTYyMTA5NDc1OQ&ixlib=rb-1.2.1&q=85
--------------------------------------------------------------------------------
https://images.unsplash.com/photo-1607670783743-4132d134b122?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8NzgxfHx0cmVlfGVufDB8fHx8MTYyMTA3MTcwNA&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1613477092384-6ada43052438?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8NzgyfHx0cmVlfGVufDB8fHx8MTYyMTA3MTcwNA&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1507144364888-ac3e6be5d3b4?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8NzgzfHx0cmVlfGVufDB8fHx8MTYyMTA3MTcwNA&ixlib=rb-1.2.1&q=85
https://images.unsplash.com/photo-1577451462333-e969f7cf989c?crop=entropy&cs=srgb&fm=jpg&ixid=MnwxMjA3fDB8MXxzZWFyY2h8Nzg0fHx0cmVlfGVufDB8fHx8MTYyMTA3MTcwNA&ixlib=rb-1.2.1&q=85

...and so on.

BeautifulSoup.find_all 无法找到所有内容- Pyhton

1 个答案: