Question

通过查看“网络”标签，我发现使用AJIAX GET在此URL检索数据：

https://shop.caremar.it/main_acquista_1_corse_00_ajax.asp?l=it&data=24/02/2019&portoP=3&portoA=5&form_url=ticket_s1_2

我已经构建了传递参数的URL，如下所示：

params = urllib.parse.urlencode({
        'data': '24/02/2019',
        'portoP': '3' ,
        'portoA': '5',
        'form_url': 'ticket_s1_2',
    })

并发出请求：

caremar_timetable_url = "https://shop.caremar.it/main_acquista_1_corse_00_ajax.asp?l=it&"
print(f"https://shop.caremar.it/main_acquista_1_corse_00_ajax.asp?l=it&{params}")
headers = {'user-agent':'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.3'}
res = requests.get(caremar_timetable_url,headers=headers, params=params)
soup = BeautifulSoup(res.text,'html.parser')
print(soup.text)

输出

https://shop.caremar.it/main_acquista_1_corse_00_ajax.asp?l=it&data=24%2F02%2F2019&portoP=7&portoA=1&form_url=ticket_s1_2
Non Ã¨ stato possibile procedere con l'acquisto del biglietto online. Si prega di riprovare

响应是来自站点的错误消息，提示它无法完成预订。如果我复制并粘贴在浏览器中创建的URL，则会得到一个带有所需数据的未样式化HTML页面。为什么会这样，我该如何克服呢？

Answer 1

数据似乎随请求返回

import requests
from bs4 import BeautifulSoup as bs
url = 'https://shop.caremar.it/main_acquista_1_corse_00_ajax.asp?l=it&data=27/02/2019&portoP=1&portoA=4&form_url=ticket_s1_2'

res = requests.get(url)
soup = bs(res.content, 'lxml')
print(soup.select_one('html'))

用请求爬AJAX页面

1 个答案: