如何解析csv文件中特定列中所有行的HTML编码文本?

时间:2019-04-23 19:42:41

标签: python python-3.x

Heres an image of how the data looks in the 'content' column

我在熊猫中加载了一个csv文件。在“内容”列中,每一行包含不同长度的html编码文本。有些就像500多个单词。我的目标是摆脱“内容”列所有行中的所有html编码。

有人可以帮助获得此代码吗?

到目前为止,我只有这个... 数据集= pd.read_csv('NuggetData.csv')

“内容”是表中的第9列(如果第一列为0),大约有17,000行。

内容列中的示例文本(顺便说一句,这不是第1行的全文,甚至更长):

第1行: <h2>A bold new toy commercial debuted last week, and it's got the internet talking.</h2><div><div data-reactroot="" class="push-wrapper--mobile" data-card="image"><img src="//i.upworthy.com/nugget/57e9536dca7292001f000008/attachments/toygif1-65977b573530a2407626f8a4aad22a4e.gif" class=""><div class="image-caption"><p>GIFs via Smyths Toys.</p></div></div></div><h2>In some ways, it was pretty standard because a boy's love for rocket ships isn't all that unique.</h2><div><div data-reactroot="" class="push-wrapper--mobile" data-card="image"><img src="//i.upworthy.com/nugget/57e953b8e2d8c7001f00002d/attachments/toygif2-6ef9ddacf2a56c63a84d773645450563.gif" class=""></div></div><h2>Neither is his love of Legos.</h2><div><div data-reactroot="" class="push-wrapper--mobile" data-card="image"><img src="//i.upworthy.com/nugget/57e95558e2d8c7002b000025/attachments/toygif4-4f0829dad2602f7dd6ed52813e6791a5.gif" class=""></div></div><h2>Plenty of boys like to (pretend to) drive motorcycles, too.</h2><div><div data-reactroot="" class="push-wrapper--mobile" data-card="image"><img src="//i.upworthy.com/nugget/57e95595ca72920034000029/attachments/toygif5-e1824fae63099796ac2947ba76ea185d.gif" class=""></div></div><h2>But ... playing dress-up as a queen in front of a crowd of cheering supporters?</h2><div><div data-reactroot="" class="push-wrapper--mobile" data-card="image"><img src="//i.upworthy.com/nugget/57e954c0e2d8c7002d00001e/attachments/toygif3-21ea60c5917fd80da817919c655a4c96.gif" class=""></div></div><p><em>That's</em> extraordinary. </p><h2>

1 个答案:

答案 0 :(得分:0)

我建议您使用BeautifulSoup(库)并列出理解内容来解析内容列。

首先,您需要了解HTML所需的内容。为了说明我在做一些假设:

  1. 您正在寻找DIV标签(findAll('div'))中的内容
  2. 假设您正在寻找上一个标记(.text)内的文本
  3. 您需要第三个DIV标签([2])中的文本
from bs4 import BeautifulSoup as bs

dataset['parsed_content'] = [bs(x,'lxml').findAll('div')[2].text for x in dataset['content']]

使用前面的代码将新列添加到数据框,但绝不修改内容。

依赖项BeautifulSoup和lxml可以使用pip安装。