Question

我想用美丽的汤来增加旅行时间：

https://betaplanner.trimet.org/map/#/?fromPlace=10255%20SW%20CANYON%20RD%3A%3A45.493227%2C-122.782138&toPlace=4809%20N%20KERBY%20AVE%3A%3A45.557817%2C-122.673216&date=2019-08-14&time=16%3A56&arriveBy=false&mode=BUS%2CTRAM%2CRAIL%2CGONDOLA%2CCAR_RENT&showIntermediateStops=true&optimize=QUICK&ignoreRealtimeUpdates=true&companies=NaN&minTransitDistance=50%25&searchTimeout=10000&onlyTransitTrips=true&ui_activeItinerary=0

问题似乎与javascript渲染有关，但是，当我尝试执行html请求时，出现了错误

对于Bs4：

 r = session.get(*linkfromabove*)
    soup = BeautifulSoup(r.content, 'html.parser')
    soup.select

对于HTML_Requests：

import requests
from requests_html import HTMLSession
session = HTMLSession()
r = session.get(linkfromabove)
r.html.render(wait=8, sleep=8)

使用BS4的实际结果：

<html lang="en">
<head>
<meta charset="utf-8"/>
<meta content="ie=edge" http-equiv="x-ua-compatible"/>
<title>TriMet MOD OTP Demo</title>
<meta content="width=device-width, initial-scale=1" name="viewport"/>
<link href="https://fonts.googleapis.com/css? 
family=Hind:300,400,500,600,700" 
rel="stylesheet"/>
<link href="main.css" rel="stylesheet"/></head>
<body>
<div id="main"></div>
<script src="bundle.js" type="text/javascript"></script></body>
</html>

来自HTML请求：

 pyppeteer.errors.NetworkError: Protocol error Target.closeTarget: Target closed.

Answer 1

如果您检查“网络”标签，您将看到一个返回json并带有您的信息的请求

import requests

r = requests.get('https://maps.trimet.org/otp_mod/plan?fromPlace=10255%20SW%20CANYON%20RD%3A%3A45.493227%2C-122.782138&toPlace=4809%20N%20KERBY%20AVE%3A%3A45.557817%2C-122.673216&date=2019-08-14&time=16%3A56&arriveBy=false&mode=BUS%2CTRAM%2CRAIL%2CGONDOLA%2CCAR_RENT%2CWALK&showIntermediateStops=true&optimize=QUICK&ignoreRealtimeUpdates=true&companies=NaN&minTransitDistance=50%25&searchTimeout=10000&onlyTransitTrips=true').json()

使用精美汤漏掉网站文字

1 个答案: