Question

我需要从苹果商店中抓取信息，我有一个带有流派和URL（{'Games'：'https://itunes.apple.com/us/genre/ios-games/id6014?mt=8'; ...}）的哈希图hashmap_genre_link，我想为每个键使用iOS应用程序（文本）和应用程序URL作为值创建另一个哈希图：games_apps：{'Pokemon Go'，'https://itunes.apple.com/us/app/pokémon-go/id1094591345?mt=8'：...}。

这是我的代码：

from bs4 import BeautifulSoup

from requests import get

links = []
ios_categories_links=[]
hashmap_genre_link ={}
url = "https://itunes.apple.com/US/genre/ios/id36"
response = get(url)
html_soup = BeautifulSoup(response.text,"html.parser")
categories_class = html_soup.find_all('div',class_="grid3-column")
# cat = categories_class.text
href = html_soup.find_all('a', href=True)
for j in href:
    # print(j['href'])
    links.append(j['href'])

#
# Hasmap initialisation : hashmap_genre_link = {"games" : "https://link_for_games_page"; etc...}
for i in links:
    if "https://itunes.apple.com/us/genre/ios" in i:
        genre = i.split("/")[5][4:] #We get the genre, without 'ios-'
        hashmap_genre_link[genre] = i
        ios_categories_links.append(i)

#print(hashmap_genre_link)


for the_key, the_value in hashmap_genre_link.items():
    #print(the_key, 'corresponds to', the_value)
    print("=======================")
    print(the_key)
    response_genre_link = get(the_value)
    html_soup_genre_link = BeautifulSoup(response_genre_link.text,"html.parser")
    genre_popular_apps_class = html_soup_genre_link.find_all('div',class_="grid3-column")
    for x in genre_popular_apps_class:
        print(x['href'])

这是输出的一部分：

=======================
games-family
<div class="grid3-column" id="selectedcontent">
<div class="column first">
<ul>
<li><a href="https://itunes.apple.com/us/app/trivia-crack/id651510680?mt=8">Trivia Crack</a> </li>
<li><a href="https://itunes.apple.com/us/app/minion-rush/id596402997?mt=8">Minion Rush</a> </li>
<li><a href="https://itunes.apple.com/us/app/draw-something-classic/id488628250?mt=8">Draw Something Classic</a> </li>

如何获取href标记中的值。（对于文本，我知道我可以使用.text

Answer 1

使用['href']来获取这些属性值是正确的想法。但是，您需要将其隔离。您的x元素包含所有带有<a>标签的href。因此，您需要另外执行x.find_all('a')，然后对它们进行迭代，并为每个href标签打印出每个<a>属性。

所以我添加了什么：

for x in genre_popular_apps_class:
        alpha = x.find_all('a')   
        for beta in alpha:
            print (beta['href'])

完整代码：

from bs4 import BeautifulSoup

from requests import get

links = []
ios_categories_links=[]
hashmap_genre_link ={}
url = "https://itunes.apple.com/US/genre/ios/id36"
response = get(url)
html_soup = BeautifulSoup(response.text,"html.parser")
categories_class = html_soup.find_all('div',class_="grid3-column")
# cat = categories_class.text
href = html_soup.find_all('a', href=True)
for j in href:
    # print(j['href'])
    links.append(j['href'])

#
# Hasmap initialisation : hashmap_genre_link = {"games" : "https://link_for_games_page"; etc...}
for i in links:
    if "https://itunes.apple.com/us/genre/ios" in i:
        genre = i.split("/")[5][4:] #We get the genre, without 'ios-'
        hashmap_genre_link[genre] = i
        ios_categories_links.append(i)

#print(hashmap_genre_link)

results_dict = {}
for the_key, the_value in hashmap_genre_link.items():
    #print(the_key, 'corresponds to', the_value)
    print("=======================")
    print(the_key)
    response_genre_link = get(the_value)
    html_soup_genre_link = BeautifulSoup(response_genre_link.text,"html.parser")
    genre_popular_apps_class = html_soup_genre_link.find_all('div',class_="grid3-column")
    for x in genre_popular_apps_class:
        alpha = x.find_all('a')
        links = [ beta['href'] for beta in alpha ]

    results_dict[the_key] = links

输出：

....
=======================
games-racing
https://itunes.apple.com/us/app/bike-race-free-style-games/id510461758?mt=8
https://itunes.apple.com/us/app/hill-climb-racing/id564540143?mt=8
https://itunes.apple.com/us/app/csr-racing/id469369175?mt=8
https://itunes.apple.com/us/app/real-racing-3/id556164008?mt=8
https://itunes.apple.com/us/app/asphalt-8-airborne/id610391947?mt=8
https://itunes.apple.com/us/app/csr-racing-2/id887947640?mt=8
https://itunes.apple.com/us/app/smashy-road-wanted/id1020119327?mt=8
https://itunes.apple.com/us/app/happy-wheels/id648668184?mt=8
https://itunes.apple.com/us/app/angry-birds-go/id642821482?mt=8
https://itunes.apple.com/us/app/need-for-speed-no-limits/id883393043?mt=8
...

在Python循环中获取href链接和文本

1 个答案: