I'm trying to scrape a website.
I'm having a problem with the bs4.BeautifulSoup
:
for i in range(len(links)):
linksRes = requests.get(links[i])
linksRes.raise_for_status()
linksSoup = bs4.BeautifulSoup(linksRes.text, 'lxml')
for e in linksSoup.find_all(href=re.compile(r'/especial/example')):
otroProducto.append(e.get('href'))
for i in range(len(otroProducto)):
detalleRes = requests.get('http://www.example.com' + otroProducto[i])
detalleRes.raise_for_status()
detalleSoup = bs4.BeautifulSoup(detalleRes.text, 'lxml')
The thing is, the first bs4.BeautifulSoup
returns a "class 'bs4.BeautifulSoup'"
as I expected, but the second one returns an empty "list"
type.
I guess it has something to do with the string concatenation I'm doing there, but I just can't figure out how to make it a BeautifulSoup type to be able to extract some info I need.