Question

尝试将数据从网页提取到表格。例如，

Block Number XXX
Building Name YYY
Street Name zzz
Pin Code 123456789

我正在尝试使用此代码以表格形式获取公司的所有详细信息...

html_doc='https://s3.amazonaws.com/todel162/test.html'

from urllib.request import urlopen
from bs4 import BeautifulSoup

soup = BeautifulSoup(urlopen(html_doc), 'html.parser')

mydivs = soup.findAll("div", {"class": "row"})

for i in mydivs:
    for x in i.findAll("div", {"class": "col-md-3 col-sm-3"}):
        print (x)
        print ('new x')

但我不知道如何在特定CSS类的HTML标记中提取文本。

Answer 1

您只想要地址信息还是整个列表？这是整个列表的代码。

html_doc='https://s3.amazonaws.com/todel162/test.html'

from urllib.request import urlopen
from bs4 import BeautifulSoup
import re

soup = BeautifulSoup(urlopen(html_doc), 'html.parser')

mydivs = soup.findAll("div", {"class": "form-group"})

for x in mydivs:
    print (re.sub( '\s+', ' ', x.text ).strip())
    print ('#############')

输出：

Information Type Other Than Individual
#############
Name Bellissimo Developers Thane Private Limited (Lodha Developers Pvt Ltd)
#############
Organization Type Company
#############
Do you have any Past Experience ? No
#############
Block Number 412, Floor 4 17G Building Name Vardhaman Chamber
#############
Street Name Cawasji Patel Road Locality Fort
#############
Land mark Horniman Circle State/UT MAHARASHTRA
#############
Division Konkan District Mumbai City
#############
Taluka Mumbai City Village Mumbai City
#############
Pin Code 400001
#############
Office Number 02261334263
#############
Website URL www.lodhagroup.com
#############

如果你想在一张桌子里（非常粗略的例子）：

...
mydivs = soup.findAll("div", {"class": "form-group"})
print ("<table>")
for x in mydivs:
    label = x.find('label')
    if label is not None:
        print ("<tr><td>" + label.text + "</td><td> ")
        label.extract()
    else:
        print ("<tr><td></td><td> ")
    print (re.sub( '\s+', ' ', x.text ).strip())
    print ('</td></tr>')
print ("</table>")

输出：

<table>
<tr><td>Information Type</td><td> 
Other Than Individual
</td></tr>
<tr><td>Name</td><td> 
Bellissimo Developers Thane Private Limited (Lodha Developers Pvt Ltd)
</td></tr>
<tr><td>Organization Type</td><td> 
Company
</td></tr>
<tr><td></td><td> 
Do you have any Past Experience ? No
</td></tr>
<tr><td>Block Number</td><td> 
412, Floor 4 17G Building Name Vardhaman Chamber
</td></tr>
<tr><td>Street Name</td><td> 
Cawasji Patel Road Locality Fort
</td></tr>
<tr><td>Land mark</td><td> 
Horniman Circle State/UT MAHARASHTRA
</td></tr>
<tr><td>Division</td><td> 
Konkan District Mumbai City
</td></tr>
<tr><td>Taluka</td><td> 
Mumbai City Village Mumbai City
</td></tr>
<tr><td>Pin Code</td><td> 
400001
</td></tr>
<tr><td>Office Number</td><td> 
02261334263
</td></tr>
<tr><td>Website URL</td><td> 
www.lodhagroup.com
</td></tr>
</table>

Answer 2

您需要的所有项目都在<div class="col-md-3 col-sm-3">标记内。并且，一般信息以外的所有项目都遵循特定格式：第一个div标记是标签，第二个是相应的文本。因此，您可以简单地找到它们，然后将它们作为标签和文本交替打印出来。

因此，{{1>}标记中提供了常规信息以外的所有内容。由于一般信息不遵循上述格式，因此您必须先单独删除它。您可以使用<div id="fldFirm">获取第一个两个标记。

limit=2

输出：

import requests
from bs4 import BeautifulSoup

r = requests.get('https://s3.amazonaws.com/todel162/test.html')
soup = BeautifulSoup(r.text, 'lxml')

gen_info = [x.text.strip() for x in soup.find('div', id='divInfoType').find_all('div', class_='col-md-3 col-sm-3', limit=2)]
print('{}: {}'.format(gen_info[0], gen_info[1]))

all_info = [x.text.strip() for x in soup.find('div', id='fldFirm').find_all('div', class_='col-md-3 col-sm-3')]

for i in range(0, len(all_info), 2):
    print('{}: {}'.format(all_info[i], all_info[i+1]))

修改

要创建数据框，您可以使用：

Information Type: Other Than Individual Name: Bellissimo Developers Thane Private Limited (Lodha Developers Pvt Ltd) Organization Type: Company Description For Other Type Organization: NA Do you have any Past Experience ?: No Block Number: 412, Floor 4 17G Building Name: Vardhaman Chamber Street Name: Cawasji Patel Road Locality: Fort Land mark: Horniman Circle State/UT: MAHARASHTRA Division: Konkan District: Mumbai City Taluka: Mumbai City Village: Mumbai City Pin Code: 400001 Office Number: 02261334263 Website URL: www.lodhagroup.com

在CSS类中提取文本

2 个答案: