Question

我有一个要清理的电影CSV文件。我正在使用Jupyter笔记本。

它具有10,000行和5列。以下是一些示例数据：

Movie Name      | Genre  | Date Released  | Length        | Rating      |
The Godfather   | Crime  | March 24, 1972 | 175           | R           |
The Avengers    | Action | May 5, 2012    | 143           | PG-13       |
The Dark Knight | Action | Crime          | July 18, 2008 | 152         | PG-13

请注意，对于“黑暗骑士”，由于有2种类型，因此行会向右移动。我想清除数据，使行变为：

The Dark Knight | Action, Crime | July 18, 2008 | 152 | PG-13

我所做的是（在Jupyter笔记本中）

import pandas as pd
path = 'movies.csv'
df = pd.read_csv(path, header=0, names=['Movie Name', 'Genre', 'Date Released','Length','Rating','Extra'])

ctrCheck = 0
months = ["January","February","March","April","May","June","July","August","September","October","November","December"]

while ctrCheck < len(df.index):
    check = str(df['Date Released'][ctrCheck])
    if any(month in check for month in months):
        replaceStr = df.loc[ctrCheck, 'Genre'] + "," + df.loc[ctrCheck, 'Date Released']
        df.loc[ctrCheck, 'Genres'] = replaceStr
        df.loc[ctrCheck, 'Date Released'] = df.loc[ctrCheck, 'Length']
        df.loc[ctrCheck, 'Length'] = df.loc[ctrCheck, 'Rating']
        df.loc[ctrCheck, 'Rating'] = df.loc[ctrCheck, 'Extra']
    ctrCheck = ctrCheck + 1

df.drop(labels='Extra', inplace=True, axis='columns')

除了遍历10,000行之外，还有更好的方法吗？

谢谢！

Answer 1

如果我正确理解，您正在寻找一种不包含显式for循环的方法，而是使用向量化pandas方法。

我们首先注意到，需要转换的行是最后一列中具有不同于Nan的值的行

因此，我可以建议以下代码：

import pandas as pd

# Name the last unnamed column
df = df.rename(columns={'Unnamed: 5': 'Extra'})

# Save the valid lines in a different dataframe
mask = (df['Extra'].isnull())
df_valid = df[mask]

# Fix the invalid lines
# Fix the Genre
df['Genre'] = df['Genre'] + ' ' + df['Date Released']
# Shift left the columns after 'Genre'
cols = df.columns[:-1]
df.drop('Date Released', axis=1, inplace=True)
df.columns = cols

# Restore valid lines
df.loc[mask, :] = df_valid

结果数据框：

        Movie Name         Genre  Date Released Length Rating
0    The Godfather         Crime  March 24 1972    175      R
1     The Avengers        Action     May 5 2012    143  PG-13
2  The Dark Knight  Action Crime   July 18 2008    152  PG-13

通知：该方法仅在每部电影的最大流派数为2时有效，如果我正确理解的话，就是这种情况：）

如何清除列移位的CSV文件？

1 个答案: