尝试将我在此处找到的代码用于类似情况,但只是获取空列表的输出。
# Import Module
from bs4 import BeautifulSoup
import requests
# Website URL
url = 'https://mubi.com/users/9167878/watchlist'
r = requests.get(url)
soup = BeautifulSoup(r.content, "lxml")
g_data = soup.find_all('h3', {"class": "film-title", 'lang': 'en'})
print(g_data)
答案 0 :(得分:0)
数据是通过 JavaScript 从外部源加载的。您可以使用此示例如何使用 requests
/json
模块加载它:
import json
import requests
url = "https://mubi.com/users/9167878/watchlist"
api_url = (
"https://mubi.com/services/api/wishes?user_id={user_id}&page=1&per_page=24"
)
user_id = url.split("/")[-2]
data = requests.get(api_url.format(user_id=user_id)).json()
# uncomment this to print all data
# print(json.dumps(data, indent=4))
for d in data:
print("{:<50} {}".format(d["film"]["title"], d["film"]["canonical_url"]))
打印:
An Affair of Love http://mubi.com/films/an-affair-of-love
I Am Self-Sufficient http://mubi.com/films/i-am-self-sufficient
Le Donne della Vucciria http://mubi.com/films/le-donne-della-vucciria
Golden Dreams http://mubi.com/films/golden-dreams
Ecce bombo http://mubi.com/films/ecce-bombo
The Solitude of Prime Numbers http://mubi.com/films/the-solitude-of-prime-numbers
De Djess http://mubi.com/films/de-djess
Salon Kitty http://mubi.com/films/salon-kitty
The Believer's Heaven http://mubi.com/films/the-believer-s-heaven
Hot Thrills and Warm Chills http://mubi.com/films/hot-thrills-and-warm-chills
Shanty Tramp http://mubi.com/films/shanty-tramp
Emerald Cities http://mubi.com/films/emerald-cities
Wild Guitar http://mubi.com/films/wild-guitar
The Burning Hell http://mubi.com/films/the-burning-hell
Satan in High Heels http://mubi.com/films/satan-in-high-heels
The Nest of the Cuckoo Birds http://mubi.com/films/the-nest-of-the-cuckoo-birds
Guilty Bystander http://mubi.com/films/guilty-bystander
Orgy of the Dead http://mubi.com/films/orgy-of-the-dead
Spring Night, Summer Night http://mubi.com/films/spring-night-summer-night
The Maidens of Fetish Street http://mubi.com/films/the-girls-on-f-street
Night Tide http://mubi.com/films/night-tide
The Guilty http://mubi.com/films/the-guilty-2018
I Am Not a Witch http://mubi.com/films/i-am-not-a-witch
The Wayward Girl http://mubi.com/films/the-wayward-girl