我有2个数据集 - 第一个包含每日天气信息,包括平均温度和加热度天数。我每天都需要一套完整的天气数据。一小段天气数据如下:
weather:
weather.station | date | temp | HDD
A | 11/30/2013 | 30 | 35
A | 12/01/2013 | 28 | 37
第二个数据集包含许多房屋的数据,每个房屋都有各自的日期范围,显示每天的燃料消耗量。例如:
home.bills:
home.id | start.date | end.date | electric.usage | weather.station
1 | 11/15/2013 | 12/14/2013 | 80 | A
1 | 12/15/2013 | 1/14/2014 | 85 | A
2 | 11/18/2013 | 12/15/2013 | 60 | A
2 | 12/16/2013 | 1/13/2014 | 57 | A
我正在寻找一种有效的方法来批量组合这两个数据集,因此我可以根据天气条件扩展home.bills信息(例如,日期范围内的平均温度和日期范围内的HDD总和)
输出可能如下所示:
output:
home.id | start.date | end.date | electric.usage | mean.temp | sum.HDD
1 | 11/15/2013 | 12/14/2013 | 80 | 32.8 | 937
1 | 12/15/2013 | 1/14/2014 | 85 | 29.7 | 1122
2 | 11/18/2013 | 12/15/2013 | 60 | 31.7 | 944
2 | 12/16/2013 | 1/13/2014 | 57 | 28.8 | 1201
有人能建议一种加入这两个数据集的方法吗?我知道如何进行总结,我不知道如何在一系列日期中加入这两个数据。
答案 0 :(得分:1)
以下是两种方法。第一个使用sqldf
但需要(??)重命名列(因为SQL不喜欢带有"。"的列名)。从概念上讲,它更简单。
weather <- structure(list(weather.station = c("A", "A"), date = c("11/30/2013 ", "12/01/2013 "), temp = c(30, 28), HDD = c(35L, 37L)), .Names = c("weather.station", "date", "temp", "HDD"), class = "data.frame", row.names = c(NA, -2L))
home.bills <- structure(list(home.id = c(1, 1, 2, 2), start.date = c(" 11/15/2013 ", " 12/15/2013 ", " 11/18/2013 ", " 12/16/2013 "), end.date = c(" 12/14/2013 ", " 1/14/2014 ", " 12/15/2013 ", " 1/13/2014 "), electric.usage = c(80, 85, 60, 57), weather.station = c("A", "A", "A", "A")), .Names = c("home.id", "start.date", "end.date", "electric.usage", "weather.station"), class = "data.frame", row.names = c(NA, -4L))
# dates need to be Dates, not character
weather$date <- as.Date(weather$date,format="%m/%d/%Y")
home.bills$start.date <- as.Date(home.bills$start.date,format="%m/%d/%Y")
home.bills$end.date <- as.Date(home.bills$end.date,format="%m/%d/%Y")
# sqldf does not like "." in column names!!!
colnames(weather) <- gsub(".","_",colnames(weather),fixed=T)
colnames(home.bills) <- gsub(".","_",colnames(home.bills),fixed=T)
library(sqldf)
sqldf("select a.*, avg(temp) as mean_temp, sum(HDD) as sum_HDD
from [home.bills] a join weather b
on b.date>=a.start_date and b.date<=a.end_date
and a.weather_station=b.weather_station
group by home_id, start_date, end_date")
# home_id start_date end_date electric_usage weather_station mean_temp sum_HDD
# 1 1 2013-11-15 2013-12-14 80 A 29 72
# 2 2 2013-11-18 2013-12-15 60 A 29 72
第二个使用foverlaps(...)
包中的data.table
功能。这非常强大且速度极快。请注意,这个解决方案与几个小时前@akrun发布的解决方案几乎相同,然后删除(我很想知道为什么?)。
# data.table solution
weather <- structure(list(weather.station = c("A", "A"), date = c("11/30/2013 ", "12/01/2013 "), temp = c(30, 28), HDD = c(35L, 37L)), .Names = c("weather.station", "date", "temp", "HDD"), class = "data.frame", row.names = c(NA, -2L))
home.bills <- structure(list(home.id = c(1, 1, 2, 2), start.date = c(" 11/15/2013 ", " 12/15/2013 ", " 11/18/2013 ", " 12/16/2013 "), end.date = c(" 12/14/2013 ", " 1/14/2014 ", " 12/15/2013 ", " 1/13/2014 "), electric.usage = c(80, 85, 60, 57), weather.station = c("A", "A", "A", "A")), .Names = c("home.id", "start.date", "end.date", "electric.usage", "weather.station"), class = "data.frame", row.names = c(NA, -4L))
library(data.table) >= 1.9.4
# convert to data.tables and convert date to Date
setDT(weather)[,date:=as.Date(date,format="%m/%d/%Y")]
setDT(home.bills)[,(2:3):=lapply(.SD,as.Date,format="%m/%d/%Y"),.SDcols=2:3]
# need start.date and end.date in weather data.table (both = date)
weather[,c("start.date","end.date"):=list(date,date)]
setkey(home.bills,weather.station,start.date,end.date)
# calaculate overlaps
result <- foverlaps(weather,home.bills,nomatch=0)
# aggregate
result[,list(mean.temp=mean(temp),sum.HDD=sum(HDD)),
by=list(home.id,start.date,end.date,electric.usage,weather.station)]
# home.id start.date end.date electric.usage weather.station mean.temp sum.HDD
# 1: 1 2013-11-15 2013-12-14 80 A 29 72
# 2: 2 2013-11-18 2013-12-15 60 A 29 72