Question

这是我试图抓取的HTML：

<span class="meta-attributes__attr-tags">
<a href="/tags/cinematic" title="cinematic">cinematic</a>, 
<a href="/tags/dissolve" title="dissolve">dissolve</a>,
<a href="/tags/epic" title="epic">epic</a>,
<a href="/tags/fly" title="fly">fly</a>,
</span>

我想获得每个href的锚文本：电影，溶解，史诗等。

这是我的代码：

url = urllib2.urlopen("http: example.com")

content = url.read()
soup = BeautifulSoup(content)

links = soup.find_all("span", {"class": "meta-attributes__attr-tags"})
for link in links:
    print link.find_all('a')['href']

如果我使用＆＃34; link.find_all＆＃34;我得到错误：TypeError：列表索引必须是整数，而不是str。

但如果我打印link.find（＆＃39; a＆＃39;）[＆＃39; href＆＃39;]我只会获得第一个。

我怎样才能得到所有这些？

Answer 1

您可以执行以下操作：

from bs4 import BeautifulSoup

content = '''
<span class="meta-attributes__attr-tags">
<a href="/tags/cinematic" title="cinematic">cinematic</a>, 
<a href="/tags/dissolve" title="dissolve">dissolve</a>,
<a href="/tags/epic" title="epic">epic</a>,
<a href="/tags/fly" title="fly">fly</a>,
</span>
'''

soup = BeautifulSoup(content)
spans = soup.find_all("span", {"class": "meta-attributes__attr-tags"})
for span in spans:
    links = span.find_all('a')
    for link in links:
        print link['href']

<强>输出

/tags/cinematic
/tags/dissolve
/tags/epic
/tags/fly

Answer 2

from bs4 import BeautifulSoup

html = """
<span class="meta-attributes__attr-tags">
<a href="/tags/cinematic" title="cinematic">cinematic</a>, 
<a href="/tags/dissolve" title="dissolve">dissolve</a>,
<a href="/tags/epic" title="epic">epic</a>,
<a href="/tags/fly" title="fly">fly</a>,
</span>
"""

soup = BeautifulSoup(html, "lxml")
spans = soup.find_all("span", {"class": "meta-attributes__attr-tags"})

for span in spans:
    for link in span.find_all('a'):
        print link.text, link['href']

另一种更昂贵的方式可能是：

from bs4 import BeautifulSoup

html = """
<span class="meta-attributes__attr-tags">
<a href="/tags/cinematic" title="cinematic">cinematic</a>,
<a href="/tags/dissolve" title="dissolve">dissolve</a>,
<a href="/tags/epic" title="epic">epic</a>,
<a href="/tags/fly" title="fly">fly</a>,
</span>
"""

soup = BeautifulSoup(html, "lxml")
links = soup.find_all("a")

for link in links:
    if 'meta-attributes__attr-tags' not in link.parent.get('class', []):
        continue

    print link.text, link['href']

Answer 3

link.find_all('a')返回带有bs4标签的列表。您可能希望按href索引每个链接。所以这可能更接近你的需求：

span = soup.find_all("span", {"class": "meta-attributes__attr-tags"})
for links in span:
    for link in links.find_all('a'):
        print(link['href'])

Answer 4

您可以使用CSS selector避免嵌套循环或在循环内进行任何其他检查：

for link in soup.select(".meta-attributes__attr-tags a[href]"):
    print(link["href"], link.get_text())

使用BeautifulSoup从span类中提取锚文本

4 个答案: