如何在具有混合数据类型值的DF ['column']上进行迭代?

时间:2018-09-12 23:00:01

标签: python pandas dataframe

我已经导入了一个excel文件,并且在pandas迭代中遇到了麻烦。 导入后数据如下所示。

Acct            Time        Typ     Name   
01/02/2018      NaN         NaN     NaN  
52              07:58:34    1       John  
53              07:58:35    1       Jack   
54              07:58:35    1       Ron  
55              07:58:35    1       Lux   
01/03/2018      NaN         NaN     NaN  
79              12:39:25    1       Mike    
01/06/2018      NaN         NaN     NaN         
371             12:15:47    1       Eric   
422             17:07:33    1       Shawn  
01/07/2018      NaN         NaN     NaN         
492             12:43:54    1       George

我要遍历第一列("Acct")的字符串,检查其是否为日期数字

我在新的数据框中创建了另一列,并尝试使用date字段填充,但是很少出现Series错误。

期望这样的输出:

Date_New        Acct        Time        Typ     Name 

01/02/2018      52          07:58:34    1       John  
01/02/2018      53          07:58:35    1       Jack   
01/02/2018      54          07:58:35    1       Ron  
01/02/2018      55          07:58:35    1       Lux   
01/03/2018      79          12:39:25    1       Mike    
01/06/2018      371         12:15:47    1       Eric   
01/06/2018      422         17:07:33    1       Shawn  
01/07/2018      492         12:43:54    1       George

如果我是pandas的新手,如果有人可以向我发送一些指导,我将不胜感激。

1 个答案:

答案 0 :(得分:2)

我建议不要迭代,而应使用pandas函数。要查找正确的日期,可以使用pd.to_datetimeAcct列转换为正确的日期,并使用参数errors = 'coerce'。非日期将变为空(NaT)。然后,使用ffill用正确的日期向前填充该列,并删除其中TimeTypName列均为NaN的列通过索引。最后,您可以对列进行重新排序:

# Find proper dates, create new column:
df['Date_New'] = pd.to_datetime(df['Acct'], errors='coerce')
# Fill non-valid dates:
df['Date_New'].ffill(inplace=True)
# Get rid of `NaN` rows:
df = df[~df[['Time','Typ', 'Name']].isnull().all(1)]
# Reorder Columns
df = df[['Date_New', 'Acct', 'Time', 'Typ', 'Name']]

>>> df
     Date_New Acct      Time  Typ    Name
1  2018-01-02   52  07:58:34  1.0    John
2  2018-01-02   53  07:58:35  1.0    Jack
3  2018-01-02   54  07:58:35  1.0     Ron
4  2018-01-02   55  07:58:35  1.0     Lux
6  2018-01-03   79  12:39:25  1.0    Mike
8  2018-01-06  371  12:15:47  1.0    Eric
9  2018-01-06  422  17:07:33  1.0   Shawn
11 2018-01-07  492  12:43:54  1.0  George

进一步的解释

为便于理解,以下是每个步骤之后的结果:

>>> df['Date_New'] = pd.to_datetime(df['Acct'], errors='coerce')
>>> df
          Acct      Time  Typ    Name   Date_New
0   01/02/2018       NaN  NaN     NaN 2018-01-02
1           52  07:58:34  1.0    John        NaT
2           53  07:58:35  1.0    Jack        NaT
3           54  07:58:35  1.0     Ron        NaT
4           55  07:58:35  1.0     Lux        NaT
5   01/03/2018       NaN  NaN     NaN 2018-01-03
6           79  12:39:25  1.0    Mike        NaT
7   01/06/2018       NaN  NaN     NaN 2018-01-06
8          371  12:15:47  1.0    Eric        NaT
9          422  17:07:33  1.0   Shawn        NaT
10  01/07/2018       NaN  NaN     NaN 2018-01-07
11         492  12:43:54  1.0  George        NaT

>>> df['Date_New'].ffill(inplace=True)
>>> df
          Acct      Time  Typ    Name   Date_New
0   01/02/2018       NaN  NaN     NaN 2018-01-02
1           52  07:58:34  1.0    John 2018-01-02
2           53  07:58:35  1.0    Jack 2018-01-02
3           54  07:58:35  1.0     Ron 2018-01-02
4           55  07:58:35  1.0     Lux 2018-01-02
5   01/03/2018       NaN  NaN     NaN 2018-01-03
6           79  12:39:25  1.0    Mike 2018-01-03
7   01/06/2018       NaN  NaN     NaN 2018-01-06
8          371  12:15:47  1.0    Eric 2018-01-06
9          422  17:07:33  1.0   Shawn 2018-01-06
10  01/07/2018       NaN  NaN     NaN 2018-01-07
11         492  12:43:54  1.0  George 2018-01-07

>>> df = df[~df[['Time','Typ', 'Name']].isnull().all(1)]
>>> df
   Acct      Time  Typ    Name   Date_New
1    52  07:58:34  1.0    John 2018-01-02
2    53  07:58:35  1.0    Jack 2018-01-02
3    54  07:58:35  1.0     Ron 2018-01-02
4    55  07:58:35  1.0     Lux 2018-01-02
6    79  12:39:25  1.0    Mike 2018-01-03
8   371  12:15:47  1.0    Eric 2018-01-06
9   422  17:07:33  1.0   Shawn 2018-01-06
11  492  12:43:54  1.0  George 2018-01-07