Question

我试图仅显示标记内的文本，例如：

<span class="listing-row__price ">$71,996</span>

我只想显示

“ 71,996美元”

我的代码是：

import requests
from bs4 import BeautifulSoup
from csv import writer

response = requests.get('https://www.cars.com/for-sale/searchresults.action/?mdId=21811&mkId=20024&page=1&perPage=100&rd=99999&searchSource=PAGINATION&showMore=false&sort=relevance&stkTypId=28880&zc=11209')

soup = BeautifulSoup(response.text, 'html.parser')

cars = soup.find_all('span', attrs={'class': 'listing-row__price'})
print(cars)

我要去哪里错了？

Answer 1

要在标签中获取文字，有两种方法，

a）使用标记的.text属性。

cars = soup.find_all('span', attrs={'class': 'listing-row__price'})
for tag in cars:
    print(tag.text.strip())

输出

$71,996
$75,831
$71,412
$75,476
....

b）使用get_text()

for tag in cars:
    print(tag.get_text().strip())

c）如果标记中只有个字符串，您也可以使用这些选项

.string
.contents[0]
next(tag.children)
next(tag.strings)
next(tag.stripped_strings)

即。

for tag in cars:
    print(tag.string.strip()) #or uncomment any of the below lines
    #print(tag.contents[0].strip())
    #print(next(tag.children).strip())
    #print(next(tag.strings).strip())
    #print(next(tag.stripped_strings))

输出：

$71,996
$75,831
$71,412
$75,476
$77,001
...

注意：

.text和.string不同。如果标记中还有其他元素，则.string返回None，而.text将返回标记内的文本。

from bs4 import BeautifulSoup
html="""
<p>hello <b>there</b></p>
"""
soup = BeautifulSoup(html, 'html.parser')
p = soup.find('p')
print(p.string)
print(p.text)

输出

None
hello there

Answer 2

print( [x.text for x in cars] )

Answer 3

实际上request不返回任何response。如我所见，响应代码为500，表示网络出现问题，您没有收到任何数据。

您缺少的是user-agent，您需要将headers与request一起发送。

import requests
import re #regex library
from bs4 import BeautifulSoup

headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36"
}

crawl_url = 'https://www.cars.com/for-sale/searchresults.action/?mdId=21811&mkId=20024&page=1&perPage=100&rd=99999&searchSource=PAGINATION&showMore=false&sort=relevance&stkTypId=28880&zc=11209'
response = requests.get(crawl_url, headers=headers )


cars = soup.find_all('span', attrs={'class': 'listing-row__price'})

for car in cars:
    print(re.sub(r'\s+', '', ''.join([car.text])))

输出

$71,412  
$75,476  
$77,001  
$77,822  
$107,271 
...

在标签BeautifulSoup中显示文本

3 个答案:

输出