Question

我每15分钟记录一次温度数据集。该文件看起来像（~50000行）

02/01/2016;05:15:00;10.800
02/01/2016;05:30:00;10.300
02/01/2016;05:45:00;9.200
02/01/2016;06:00:00;9.200
02/01/2016;06:15:00;8.900
02/01/2016;06:30:00;8.900
02/01/2016;06:45:00;9.400
02/01/2016;07:00:00;9.000
02/01/2016;07:15:00;9.200
02/01/2016;07:30:00;11.100
02/01/2016;07:45:00;13.000
02/01/2016;08:00:00;14.400
02/01/2016;08:15:00;15.600

我的目标是计算每日最低/最高，所以这里是我的代码

# load dataframe
with open(intraday_file_path, "r") as fl:
    df_intraday = pd.read_csv(fl,
                              **load_args
                              )

df_daily = df_intraday.groupby(df_intraday[0])
df_daily = df_daily.aggregate({0:np.max})

df_daily.index.names = [0]
df_daily.reset_index(level=[0], inplace=True)

df_daily.sort_values(by=[0], inplace=True)
df_daily.drop_duplicates(subset=0,
                         keep="first",
                         inplace=True)

daily_name = "daily_%s" %(intraday_file_name,)
daily_path = os.getcwd() + "\\" + daily_name

df_daily = df_daily[0, 1]

with open(daily_path, "w") as fl:
    df_daily.to_csv(fl,
                    **save_args
                    )

但是一旦温度低于10°C，输出就很奇怪。例如，对于02/01/2016，代码输出为9.4°C？！

有什么想法吗？

Answer 1

您的数据在最后一栏中没有数字存在问题。

解决方案是使用to_numeric将错误数据转换为NaN s：

为了更好地使用DataFrame，可以为列名添加参数names到read_csv。

import pandas as pd
from pandas.compat import StringIO
temp=u"""02/01/2016;05:15:00;10.800
02/01/2016;05:30:00;10.300
02/01/2016;05:45:00;9.200
02/01/2016;06:00:00;9.200
02/01/2016;06:15:00;8.900
02/01/2016;06:30:00;8.900
02/01/2016;06:45:00;9.400
03/01/2016;07:00:00;9.000
03/01/2016;07:15:00;9.200
03/01/2016;07:30:00;11.100
04/01/2016;07:45:00;13.000
04/01/2016;08:00:00;14.400
04/01/2016;08:15:00;a"""
#after testing replace 'StringIO(temp)' to 'filename.csv'
df_intraday = pd.read_csv(StringIO(temp), 
                          sep=";", 
                          names=['date','time','val'], 
                          parse_dates=[0])
print (df_intraday)
         date      time     val
0  2016-02-01  05:15:00  10.800
1  2016-02-01  05:30:00  10.300
2  2016-02-01  05:45:00   9.200
3  2016-02-01  06:00:00   9.200
4  2016-02-01  06:15:00   8.900
5  2016-02-01  06:30:00   8.900
6  2016-02-01  06:45:00   9.400
7  2016-03-01  07:00:00   9.000
8  2016-03-01  07:15:00   9.200
9  2016-03-01  07:30:00  11.100
10 2016-04-01  07:45:00  13.000
11 2016-04-01  08:00:00  14.400
12 2016-04-01  08:15:00       a

df_daily = df_intraday.groupby('date', as_index=False)['val'].max()
print (df_daily)
        date    val
0 2016-02-01  9.400
1 2016-03-01  9.200
2 2016-04-01      a

#check dtypes -  object is obviusly string
print (df_intraday['val'].dtypes)
object

df_intraday['val'] = pd.to_numeric(df_intraday['val'], errors='coerce')
print (df_intraday)
         date      time   val
0  2016-02-01  05:15:00  10.8
1  2016-02-01  05:30:00  10.3
2  2016-02-01  05:45:00   9.2
3  2016-02-01  06:00:00   9.2
4  2016-02-01  06:15:00   8.9
5  2016-02-01  06:30:00   8.9
6  2016-02-01  06:45:00   9.4
7  2016-03-01  07:00:00   9.0
8  2016-03-01  07:15:00   9.2
9  2016-03-01  07:30:00  11.1
10 2016-04-01  07:45:00  13.0
11 2016-04-01  08:00:00  14.4
12 2016-04-01  08:15:00   NaN

print (df_intraday['val'].dtypes)
float64

#simplier way for aggregating max
df_daily = df_intraday.groupby('date', as_index=False)['val'].max()
print (df_daily)
        date   val
0 2016-02-01  10.8
1 2016-03-01  11.1
2 2016-04-01  14.4

Answer 2

仅供参考，有resample选项。时间序列的便利工具。

import pandas as pd


df = pd.read_table('sample.txt', header=None, sep=';')
df.columns=['date', 'time', 'temp']

df['datetime'] = pd.to_datetime(df['date'] + ' ' + df['time'])
df['temp'] = df['temp'].astype(float)  # dtypes should be float as jezrael mentioned.

df = df.set_index('datetime')[['temp']]
df = pd.concat([df.resample('1D', how=min), 
                df.resample('1D', how=max)], axis=1)

df.columns = ['temp_min', 'temp_max']
print(df)

结果

            temp_min  temp_max
datetime                      
2016-02-01       8.9      15.6

获得每日最大值会产生奇怪的结果

2 个答案: