Question

我是数据科学的学生，但到目前为止几乎没有代码经验。

我的问题是：如何从字符串中获取dicts列表，该字符串已经以dicts列表的形式出现，但是pandas将其视为字符串？

这是数据集（学分）： https://www.kaggle.com/tmdb/tmdb-movie-metadata/data

在“＆＃39; cast＆＃39;和＆＃39;船员＆＃39;我有这样的细胞：

[
{"credit_id": "52fe420dc3a36847f800012d", "department": "Directing", "gender": 1, "id": 3110, "job": "Director", "name": "Allison Anders"}, 
{"credit_id": "52fe420dc3a36847f80001c9", "department": "Writing", "gender": 1, "id": 3110, "job": "Writer", "name": "Allison Anders"}
]

（显然每个细胞都有几十个序列）

我的主要问题是，在我加载文件并创建了一个数据框之后，这两个列的单元格（演员和工作人员）被pandas视为字符串，而不是作为一个字典列表，所以我无法执行我需要的操作。

creditsB = pd.read_csv('folder\\tmdb_5000_credits.csv')
creditsDF = pd.DataFrame(creditsB)
type(creditsDF.loc[0,'crew'])
# str

如果我尝试在其上应用list（），它只会创建一个单个字符列表。

dct = list(creditsDF.loc[0,'crew'])
dct
 # output:
 ['[',
 '{',
 '"',
 'c',
 'r',
 'e',
 # and so on

如何让python理解它实际上是一个dicts列表，并对其进行处理？

我必须为每部电影做一些像＆＃34;这样的基本操作，计算演员的数量＆＃34;或者＆＃34;对于每部电影，计算导演的数量＆＃34;。如果我刚刚解决了这个大问题，那将非常容易。

提前感谢您的帮助！

Answer 1

您必须在列表

中附加dict

 movies = [ {"credit_id": "52fe420dc3a36847f800012d", "department": "Directing", "gender": 1, "id": 3110, "job": "Director", "name": "Allison Anders"}, {"credit_id": "52fe420dc3a36847f80001c9", "department": "Writing", "gender": 1, "id": 3110, "job": "Writer", "name": "Allison Anders"} ]

    for movie in movies:
        print movie["name"]

    # count movies in list
    print len(movies)

Answer 2

尝试ast.literal_eval：

import ast

text = '''
[
{"credit_id": "52fe420dc3a36847f800012d", "department": "Directing", "gender": 1, "id": 3110, "job": "Director", "name": "Allison Anders"}, 
{"credit_id": "52fe420dc3a36847f80001c9", "department": "Writing", "gender": 1, "id": 3110, "job": "Writer", "name": "Allison Anders"}
]
'''

dicts = ast.literal_eval(text)
# [{'name': 'Allison Anders', 'department': 'Directing', 'credit_id': '52fe420dc3a36847f800012d', 'gender': 1, 'job': 'Director', 'id': 3110}, 
# {'name': 'Allison Anders', 'department': 'Writing', 'credit_id': '52fe420dc3a36847f80001c9', 'gender': 1, 'job': 'Writer', 'id': 3110}]
print(len(dicts))
# 2
print(dicts[0]['department'])
# Directing

要进行有效的应用更改，请尝试apply：

df['col'] = df['col'].apply(lambda x: ast.literal_eval(x))

从词典中提取所需的字段：

dicts = ast.literal_eval(text)
[d['department'] for d in dicts]
# ['Directing', 'Writing']

Answer 3

所以你有字典列表，但它们在数据框中显示为字符串。这非常效率低下。您应该致力于改进工作流上游，以便直接将字典读入Python。

但是，根据您拥有的内容，您可以使用ast.literal_eval从字面上读取字符串。然后输入pd.DataFrame。这是有效的，因为pd.DataFrame直接接受字典列表。

进入数据框后，您可以：

通过len(df.index)计算字典数。
使用Pandas布尔索引进行过滤，例如df.loc[df['job'] == 'Director', 'name']将过滤导演姓名。

以下是一个例子：

import pandas as pd
from itertools import chain
from ast import literal_eval

s = pd.Series(['[{"credit_id": "52fe420dc3a36847f800012d", "department": "Directing", "gender": 1, "id": 3110, "job": "Director", "name": "Allison Anders"},{"credit_id": "52fe420dc3a36847f80001c9", "department": "Writing", "gender": 1, "id": 3110, "job": "Writer", "name": "DEF GHI"}]',
               '[{"credit_id": "52fe420dc3a36847f800012e", "department": "Costume", "gender": 0, "id": 4110, "job": "Dresser", "name": "A B"},{"credit_id": "52fe420dc3a36847f80001c8", "department": "Videography", "gender": 1, "id": 3111, "job": "Other", "name": "Joe Smith"}]',
               '[{"credit_id": "52fe420dc3a36847f800012f", "department": "Music", "gender": 1, "id": 5110, "job": "Composer", "name": "C D"},{"credit_id": "52fe420dc3a36847f80001c7", "department": "Production", "gender": 0, "id": 3112, "job": "Writer", "name": "Ben Andrews"}]'])

print(s)

# 0    [{"credit_id": "52fe420dc3a36847f800012d", "de...
# 1    [{"credit_id": "52fe420dc3a36847f800012e", "de...
# 2    [{"credit_id": "52fe420dc3a36847f800012f", "de...
# dtype: object

chained = chain.from_iterable(literal_eval(i) for i in s)

df = pd.DataFrame(list(chained))

print(df)

#                   credit_id   department  gender    id       job  \
# 0  52fe420dc3a36847f800012d    Directing       1  3110  Director   
# 1  52fe420dc3a36847f80001c9      Writing       1  3110    Writer   
# 2  52fe420dc3a36847f800012e      Costume       0  4110   Dresser   
# 3  52fe420dc3a36847f80001c8  Videography       1  3111     Other   
# 4  52fe420dc3a36847f800012f        Music       1  5110  Composer   
# 5  52fe420dc3a36847f80001c7   Production       0  3112    Writer   

#              name  
# 0  Allison Anders  
# 1         DEF GHI  
# 2             A B  
# 3       Joe Smith  
# 4             C D  
# 5     Ben Andrews

Python将dicts列表视为字符串：如何解析？

3 个答案: