Question

我想使用pandas dataframe，列名称 - Product Title和populate t来表示数据。

例如：

产品标题

Marvel：电影收藏

Marvel

Diney Movie等......

import requests
from bs4 import BeautifulSoup
import csv
import pandas as pd

r= requests.get("http://www.walmart.com/search/?query=marvel&cat_id=4096_530598")
r.content
soup = BeautifulSoup(r.content)

g_data = soup.find_all("div", {"class" : "tile-conent"})
g_price = soup.find_all("div",{"class" : "item-price-container"})
g_star = soup.find_all("div",{"class" : "stars stars-small tile-row"})

for product_title in g_data:
   a_product_title = product_title.find_all("a","js-product-title")
   for text_product_title in a_product_title : 
      t = text_product_title.text  
      print t

期望输出 -

Product Title : 


Marvel Heroes: Collection      
Marvel: Guardians Of The Galaxy (Widescreen)    
Marvel Complete Giftset (Widescreen)    
Marvel's The Avengers (Widescreen)    
Marvel Knights: Wolverine Versus Sabretooth - Reborn (Widescreen)    
Superheroes Collection: The Incredible Hulk Returns / The Trial Of The Incredible Hulk / How To Draw Comics     The Marvel Way (Widescreen)
Marvel: Iron Man & Hulk - Heroes United (Widescreen)    
Marvel's The Avengers (DVD + Blu-ray) (Widescreen)     
Captain America: The Winter Soldier (Widescreen)    
Iron Man 3 (DVD + Digital Copy) (Widescreen)    
Thor: The Dark World (Widescreen)    
Spider-Man (2-Disc) (Special Edition) (Widescreen)    
Elektra / Fantastic Four / Daredevil (Director's Cut) / Fantastic Four 2: Rise Of The Silver Surfer
Spider-Man / Spider-Man 2 / Spider-Man 3 (Widescreen)    
Spider-Man 2 (Widescreen)    
The Punisher (Extended Cut) (Widescreen)    
DC Showcase: Superman / Shazam!: The Return Of The Black Adam
Ultimate Avengers: The Movie (Widescreen)    
The Next Avengers: Heroes Of Tomorrow (Widescreen)    
Ultimate Avengers 1 & 2 (Blu-ray) (Widescreen)

我累了附加功能并加入但是它工作了..在pandas数据帧中我们有任何特定的功能吗？

所需的输出应该是使用Pandas数据帧的结果。

Answer 1

这会让你开始，这会将所有标题提取到一个字典中（为方便起见，我使用了defaultdict）：

In [163]:

from collections import defaultdict
data=defaultdict(list)
for product_title in g_data:
    a_product_title = product_title.find_all("a","js-product-title")
    for text_title in a_product_title:
        data['Product title'].append(text_title.text)


df = pd.DataFrame(data)
df
Out[163]:
                                        Product title
0                           Marvel Heroes: Collection
1        Marvel: Guardians Of The Galaxy (Widescreen)
2                Marvel Complete Giftset (Widescreen)
3                  Marvel's The Avengers (Widescreen)
4   Marvel Knights: Wolverine Versus Sabretooth - ...
5   Superheroes Collection: The Incredible Hulk Re...
6   Marvel: Iron Man & Hulk - Heroes United (Wides...
7   Marvel's The Avengers (DVD + Blu-ray) (Widescr...
8    Captain America: The Winter Soldier (Widescreen)
9        Iron Man 3 (DVD + Digital Copy) (Widescreen)
10                  Thor: The Dark World (Widescreen)
11  Spider-Man (2-Disc) (Special Edition) (Widescr...
12  Elektra / Fantastic Four / Daredevil (Director...
13  Spider-Man / Spider-Man 2 / Spider-Man 3 (Wide...
14                          Spider-Man 2 (Widescreen)
15           The Punisher (Extended Cut) (Widescreen)
16  DC Showcase: Superman / Shazam!: The Return Of...
17          Ultimate Avengers: The Movie (Widescreen)
18  The Next Avengers: Heroes Of Tomorrow (Widescr...
19     Ultimate Avengers 1 & 2 (Blu-ray) (Widescreen)

因此，您可以修改此脚本以将价格和参与者添加为数据字典的键，然后从结果字典中构造df，这将比一次附加一行更好

熊猫数据帧

1 个答案: