我有一个长数据框，其中包含来自桅杆的气象数据。它包含在不同高度（data$value）data$param）中同时拍摄的观测值（data$z） >
我试图通过$time有效地对这些数据进行切片，然后将函数应用于所有收集的数据。通常，函数一次应用于单个$param（即，我对风速应用不同的函数而不是空气温度）。
我目前的方法是使用data.frame和ddply。
如果我想获得所有的风速数据，我会运行：
# find good data ----
df <- data[((data$param == "wind speed") &
!is.na(data$value)),]
然后我使用df
ddply()上运行我的功能
df.tav <- ddply(df,
.(time),
function(x) {
y <-data.frame(V1 = sum(x$value) + sum(x$z),
V2 = sum(x$value) / sum(x$z))
return(y)
})
通常V1和V2是对其他功能的调用。这些只是一些例子。我确实需要在相同的数据上运行多个函数。
我目前的方法非常慢。我没有对它进行基准测试，但它足够慢，我可以去喝咖啡，然后在一年之前回来处理数据。
我有订单（百）塔要处理，每个塔都有一年的数据和10-12个高度，所以我正在寻找更快的东西。
data <- structure(list(time = structure(c(1262304600, 1262304600, 1262304600,
1262304600, 1262304600, 1262304600, 1262304600, 1262304600, 1262304600,
1262304600, 1262304600, 1262304600, 1262304600, 1262304600, 1262304600,
1262304600, 1262304600, 1262304600, 1262304600, 1262304600, 1262304600,
1262304600, 1262304600, 1262304600, 1262304600, 1262304600, 1262304600,
1262304600, 1262304600, 1262304600, 1262304600, 1262304600, 1262304600,
1262305200, 1262305200, 1262305200, 1262305200, 1262305200, 1262305200,
1262305200), class = c("POSIXct", "POSIXt"), tzone = ""), z = c(0,
0, 0, 100, 100, 100, 120, 120, 120, 140, 140, 140, 160, 160,
160, 180, 180, 180, 200, 200, 200, 40, 40, 40, 50, 50, 50, 60,
60, 60, 80, 80, 80, 0, 0, 0, 100, 100, 100, 120), param = c("temperature",
"humidity", "barometric pressure", "wind direction", "turbulence",
"wind speed", "wind direction", "turbulence", "wind speed", "wind direction",
"turbulence", "wind speed", "wind direction", "turbulence", "wind speed",
"wind direction", "turbulence", "wind speed", "wind direction",
"turbulence", "wind speed", "wind direction", "turbulence", "wind speed",
"wind direction", "turbulence", "wind speed", "wind direction",
"turbulence", "wind speed", "wind direction", "turbulence", "wind speed",
"temperature", "barometric pressure", "humidity", "wind direction",
"wind speed", "turbulence", "wind direction"), value = c(-2.5,
41, 816.9, 248.4, 0.11, 4.63, 249.8, 0.28, 4.37, 255.5, 0.32,
4.35, 252.4, 0.77, 5.08, 248.4, 0.65, 3.88, 313, 0.94, 6.35,
250.9, 0.1, 4.75, 253.3, 0.11, 4.68, 255.8, 0.1, 4.78, 254.9,
0.11, 4.7, -3.3, 816.9, 42, 253.2, 2.18, 0.27, 229.5)), .Names = c("time",
"z", "param", "value"), row.names = c(NA, 40L), class = "data.frame")
答案 0 :(得分：14)
使用data.table：
library(data.table)
dt = data.table(data)
setkey(dt, param) # sort by param to look it up fast
dt[J('wind speed')][!is.na(value),
list(sum(value) + sum(z), sum(value)/sum(z)),
by = time]
# time V1 V2
#1: 2009-12-31 18:10:00 1177.57 0.04209735
#2: 2009-12-31 18:20:00 102.18 0.02180000
如果你想为每个参数应用不同的函数，这里有一个更统一的方法。
# make dt smaller because I'm lazy
dt = dt[param %in% c('wind direction', 'wind speed')]
# now let's start - create another data.table
# that will have param and corresponding function
fns = data.table(p = c('wind direction', 'wind speed'),
fn = c(quote(sum(value) + sum(z)), quote(sum(value) / sum(z))),
key = 'p')
fns
p fn
1: wind direction <call> # the fn column contains functions
2: wind speed <call> # i.e. this is getting fancy!
# now we can evaluate different functions for different params,
# sliced by param and time
dt[!is.na(value), {param; eval(fns[J(param)]$fn[[1]], .SD)},
by = list(param, time)]
# param time V1
#1: wind direction 2009-12-31 18:10:00 3.712400e+03
#2: wind direction 2009-12-31 18:20:00 7.027000e+02
#3: wind speed 2009-12-31 18:10:00 4.209735e-02
#4: wind speed 2009-12-31 18:20:00 2.180000e-02
P.S。我认为我必须在param eval之前以某种方式使用eval才能使dt[!is.na(value), eval(fns[J(param)]$fn[[1]], .SD), by = list(param, time)]
工作是一个错误。
更新：自version 1.8.11起，此错误已修复，以下内容有效：
{{1}}
答案 1 :(得分：9)
使用dplyr。它仍在开发中，但它比plyr快得多：
# devtools::install_github(dplyr)
library(dplyr)
windspeed <- subset(data, param == "wind speed")
daily <- group_by(windspeed, time)
summarise(daily, V1 = sum(value) + sum(z), V2 = sum(value) / sum(z))
dplyr的另一个优点是你可以使用数据表作为后端，而不必了解data.table的特殊语法：
library(data.table)
daily_dt <- group_by(data.table(windspeed), time)
summarise(daily_dt, V1 = sum(value) + sum(z), V2 = sum(value) / sum(z))
（带有数据帧的dplyr比plyr快20-100倍，带有data.table的dplyr大约快10倍）。 dplyr远不如data.table那么简洁，但是它具有数据分析的每个主要任务的功能，我发现这使得代码更容易理解 - 你几乎能够读取一系列dplyr操作给别人和让他们了解发生了什么。
如果您想对每个变量执行不同的摘要，建议您将数据结构更改为“tidy”：
library(reshape2)
data_tidy <- dcast(data, ... ~ param)
daily_tidy <- group_by(data_tidy, time)
summarise(daily_tidy,
mean.pressure = mean(`barometric pressure`, na.rm = TRUE),
sd.turbulence = sd(`barometric pressure`, na.rm = TRUE)
)