我正在尝试抓取Traveloka.com网站并成功获取评论客户的数据。但是,问题出在1页上。我需要在(第1,2,3 ....页以及更多)上获得评论客户的所有数据。我使用beautifulsoup4。我试图修改代码并查看教程,但仍然无法正常工作。请帮助我。这是我下面的代码。
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
my_url = 'https://www.traveloka.com/id-id/hotel/indonesia/horison-ultima-bandung-2000000081026?spec=26-05-2019.27-05-2019.1.1.HOTEL.2000000081026.Horison%20Ultima%20Bandung.2&prevSearchId=1634474608622074440&loginPromo=1&contexts=%7B%7D'
page_soup = soup(page_html, "html.parser")
containers = page_soup.findAll("div",{"class":"_2K0Zb _278Mz"}) #div reviews
对于容器中的容器:
username_container = container.findAll("div",{"class":"css-76zvg2 r-1inkyih r-b88u0q"}) # review (username)
username = username_container[0].text
tanggal_container = container.findAll("div",{"class":"css-76zvg2 r-1ud240a r-1b43r93 r-b88u0q r-1d4mawv r-tsynxw"}) # review (tanggal)
tanggal = tanggal_container[0].text
deskripsi_container = container.findAll("div",{"class":"css-1dbjc4n r-1wzrnnt"}) # review (deskripsi)
deskripsi = deskripsi_container[0].text
print("username : " + username)
print("tanggal : " + tanggal)
print("deskripsi : " + deskripsi)