Question

I've used pandas.read_csv to load in a file.

I've stored the file into a variable. The first column is a series of numbers separated by a comma (,) I want to split these numbers, and put each number to a new column.

I can't seem to find the write functionality for pandas.dataframe.

Side Note I would prefer a different library for loading in my file, but pandas provides some other different functionality which I need.

My Code:

Data = pandas.read_csv(pathFile,header=None)

doing: print Data gives me:

   0                          1         2          ...
0 [2014, 8, 26, 5, 30, 0.0]   0         0.25       ...

(as you can see its a date)

Question: How to split/separate each number and save it in a new array

p.s. I'm trying to achieve the same thing the matlab method datevec() does

Answer 1

如果CSV数据看起来像

"[2014, 8, 26, 5, 30, 0.0]",0,0.25

然后

import pandas as pd
import json

df = pd.read_csv('data', header=None)
dates, df = df[0], df.iloc[:, 1:]
df = pd.concat([df, dates.apply(lambda x: pd.Series(json.loads(x)))], axis=1,
               ignore_index=True)
print(df)

产量

   0     1     2  3   4  5   6  7
0  0  0.25  2014  8  26  5  30  0

将值解析为数值。

工作原理：

dates, df = df[0], df.iloc[:, 1:]

剥离第一列，并将df重新分配给DataFrame的其余部分：

In [217]: dates
Out[217]: 
0    [2014, 8, 26, 5, 30, 0.0]
Name: 0, dtype: object

dates包含字符串：

In [218]: dates.iloc[0]
Out[218]: '[2014, 8, 26, 5, 30, 0.0]'

我们可以使用json.loads将这些转换为列表：

In [219]: import json

In [220]: json.loads(dates.iloc[0])
Out[220]: [2014, 8, 26, 5, 30, 0.0]

In [221]: type(json.loads(dates.iloc[0]))
Out[221]: list

我们可以使用dates：

为apply的每一行执行此操作

In [222]: dates.apply(lambda x: pd.Series(json.loads(x)))
Out[222]: 
      0  1   2  3   4  5
0  2014  8  26  5  30  0

通过上面的lambda，返回一个系列，apply将返回一个DataFrame，系列的索引成为DataFrame的列索引。

现在我们可以使用pd.concat将此DataFrame与df连接起来：

In [228]: df = pd.concat([df, dates.apply(lambda x: pd.Series(json.loads(x)))], axis=1, ignore_index=True)

In [229]: df
Out[229]: 
   0     1     2  3   4  5   6  7
0  0  0.25  2014  8  26  5  30  0

In [230]: df.dtypes
Out[230]: 
0      int64
1    float64
2    float64
3    float64
4    float64
5    float64
6    float64
7    float64
dtype: object

Answer 2

怎么样

df
#                   datestr
#0  2014, 8, 26, 5, 30, 0.0
#1  2014, 8, 26, 5, 30, 0.0
#2  2014, 8, 26, 5, 30, 0.0
#3  2014, 8, 26, 5, 30, 0.0
#4  2014, 8, 26, 5, 30, 0.0

# each entry is a string
df.datestr[0]
#'2014, 8, 26, 5, 30, 0.0'

然后

date_order = ('year', 'month','day','hour','minute','sec') # order matters here, should match the datestr column 

for i,col in enumerate( date_order):
    df[col] = df.datestr.map( lambda x: x.split(',')[i].strip() )

#df
#                   datestr  year month day hour minute  sec
#0  2014, 8, 26, 5, 30, 0.0  2014     8  26    5     30  0.0
#1  2014, 8, 26, 5, 30, 0.0  2014     8  26    5     30  0.0
#2  2014, 8, 26, 5, 30, 0.0  2014     8  26    5     30  0.0
#3  2014, 8, 26, 5, 30, 0.0  2014     8  26    5     30  0.0
#4  2014, 8, 26, 5, 30, 0.0  2014     8  26    5     30  0.0

Python pandas.read_csv split column into multiple new columns using comma to separate

2 个答案: