在python中刮表值

时间:2018-01-23 20:44:07

标签: python html web-scraping beautifulsoup

SO的新手,并且使用beautifulsoup从网站上抓桌子时遇到了一些困难。

表格的源html就是这样的(每个艺术家/歌曲/专辑都重复出现了恶心):

<td class="subject">
    <p title="song">song</p>
    <p class="singer" title="artist | album">artist<span class="bar">|</span>album</p>
</td>

我正在尝试使用所有这些信息创建输出文件。我使用的代码是:

with open('output.txt', 'w', encoding='utf-8') as f:
for tr in soup.find_all('tr')[1:]:
    tds = tr.find_all('td')
    f.write("Information: %s" % tds[3].text)

给我一​​个像这样的输出:

Information: 
song
singer | album

如何更改此设置以将其全部放在一行上,并将其正确分开?理想情况下,我的输出应该是这样的:

Song Title: song
Artist: singer
Album Name: album

2 个答案:

答案 0 :(得分:2)

我认为你很接近,你只需要处理tds的结果。我会做以下事情:

from bs4 import BeautifulSoup
b = BeautifulSoup(html, 'lxml')

html = """<td class="subject">
    <p title="song">song</p>
    <p class="singer" title="artist | album">artist<span class="bar">|</span>album</p>
</td>"""

tds = b.find_all('td')
data = tds[0]

t = data.text.split('\n')
song = t[1]
artist_album = t[2].split('|')
artist = artist_album[0]
album = artist_album[1]
print("Song:", song)
print("Artist:", artist)
print("Album:", album)

这应该给你:

Song: song
Artist: artist
Album: album

答案 1 :(得分:1)

您可以将正则表达式与BeautifulSoup

一起使用
from bs4 import BeautifulSoup as soup
import re
s = """
<td class="subject">
<p title="song">song</p>
<p class="singer" title="artist | album">artist<span class="bar">|</span>album</p>
 </td>
"""
s = soup(s, 'lxml')

data = [list(filter(None, c))[0] for c in [re.findall('title="song">(.*?)</p>|album">(.*?)<span class="bar">|</span>(.*?)</p>', str(i)) for i in s.find_all('td', {'class':'subject'})][0]]
for i in zip(['Song', 'Artist', 'Album'], data):
   print('{}: {}'.format(*i))

输出:

Song: song
Artist: artist
Album: album