Question

因此，我想从中删除总监。但正如我所看到的那样，我知道这部电影的丹尼·博伊尔和洛夫·丹丹有两位导演。但是，如果我使用find_all（'a'），就无法获得它，那么它也将采用像Dev Patel，Freida Pinto这样的演员的名字。

我不能使用find_all（'a'）[1]和find_all（'a'）[2]，因为对于其他电影，可能只有一个导演。唯一将演员与导演区分开来的是带有类ghost的span标签。假设可能有一位，两位或三位董事，我应该如何删除这些数据。

<p class="">
             Directors:
             <a href="/name/nm0000965/">
              Danny Boyle
             </a>
             ,
             <a href="/name/nm0849164/">
              Loveleen Tandan
             </a>
             <span class="ghost">
              |
             </span>
             Stars:
             <a href="/name/nm2353862/">
              Dev Patel
             </a>
             ,
             <a href="/name/nm2951768/">
              Freida Pinto
             </a>
             ,
             <a href="/name/nm0795661/">
              Saurabh Shukla
             </a>
             ,
             <a href="/name/nm0438463/">
              Anil Kapoor
             </a>
            </p>

页面的网址是： https://www.imdb.com/search/title/?count=100&groups=oscar_best_picture_winners&sort=year%2Cdesc&ref_=nv_ch_osc

Answer 1

这对您有帮助：

from bs4 import BeautifulSoup

html = """
<p class="">
             Directors:
             <a href="/name/nm0000965/">
              Danny Boyle
             </a>
             ,
             <a href="/name/nm0849164/">
              Loveleen Tandan
             </a>
             <span class="ghost">
              |
             </span>
             Stars:
             <a href="/name/nm2353862/">
              Dev Patel
             </a>
             ,
             <a href="/name/nm2951768/">
              Freida Pinto
             </a>
             ,
             <a href="/name/nm0795661/">
              Saurabh Shukla
             </a>
             ,
             <a href="/name/nm0438463/">
              Anil Kapoor
             </a>
            </p>
""" #The html code provided by you

soup = BeautifulSoup(html,'html5lib')

p_tag = soup.find('p')

span = p_tag.find('span',class_ = "ghost")

prev = list(span.previous_siblings) #Finds all the tags before the span tag with class ghost and converts them into a list

prev = [str(x) for x in prev]

prev = ''.join(prev) #Converts the list to a string

soup2 = BeautifulSoup(prev,'html5lib') #Creates a new BeautifulSoup object with the newly formed string

a_tags = soup2.find_all('a')

for a in a_tags:
    txt = a.text.strip()
    print(txt)

输出：

Loveleen Tandan
Danny Boyle

希望这会有所帮助！

Answer 2

我就把它留在这里

import requests
from bs4 import BeautifulSoup


url = 'https://www.imdb.com/search/title/?count=100&groups=oscar_best_picture_winners&sort=year%2Cdesc&ref_=nv_ch_osc'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

directors_and_stars = soup.find_all(text=lambda t: 'Director' in t)

for d in directors_and_stars:
    movie_name = d.find_previous('h3').a.get_text(strip=True)
    directors = [t.strip() for t in d.find_previous('p').find_all(text=True)[1:] if t.strip() and t.strip() != ',']
    directors = directors[:directors.index('|')]
    print('{:<50} {}'.format(movie_name, directors))

打印：

Parazit                                            ['Bong Joon Ho']
Zelená kniha                                       ['Peter Farrelly']
The Shape of Water                                 ['Guillermo del Toro']
Moonlight                                          ['Barry Jenkins']
Spotlight                                          ['Tom McCarthy']
Birdman or (The Unexpected Virtue of Ignorance)    ['Alejandro G. Iñárritu']
12 rokov otrokom                                   ['Steve McQueen']
Argo                                               ['Ben Affleck']
The Artist                                         ['Michel Hazanavicius']
Králova rec                                        ['Tom Hooper']
Slumdog Millionaire                                ['Danny Boyle', 'Loveleen Tandan']
Smrt' caká vsade                                   ['Kathryn Bigelow']
Táto krajina nie je pre starých                    ['Ethan Coen', 'Joel Coen']

...and so on.

使用Beautiful Soup进行Web抓取，无需类即可抓取多个元素

2 个答案: