如何格式化r中具有重复日期的列

时间:2014-09-06 11:07:46

标签: r datetime xts

我有一个相当大的数据集保存在.txt文件中,我使用fread()中的data.table读入我的环境。我想组织数据,使它看起来像这样:

# Desired output: Column headers = [ Date, Company Names, RET, RETX,PRC, vwretd, ewrtd, sprtrn]
# rows contain stock return data... (I made up the values below to save time, but the format must remain).


   Date        AMERICAN CAR & FDRY CO   ALASKA JUNEAU GOLD MNG C      RET       RETX   vwretd ...
1925-12-31          0.0432                    0.0231                 0.0032     0.053   ...
1926-01-02          0.0210                    0.0133                 0.0124     0.003   ...
1926-01-04          0.0324                    0.0131                 0.0134     0.023   ...
...                 ...                      ...                      ...        ...    ...

唯一的问题是我遇到了麻烦,因为我的crsp$date列有重复的日期 (见可再生数据),因为数据似乎是row binded

以下是使用fread()

导入数据时运行的内容
crsp <- fread("/Volumes/share/CRSP/CRSP_RETS/1925-1930.txt", sep="\t", header=TRUE)
crsp <- as.data.frame(crsp)

导入后,我运行以下内容以格式化date

crsp$DATE <- as.Date(as.character(crsp$date),format="%Y%m%d")

然后我尝试转换为XTS

data <- xts(crsp[,3:11], crsp$DATE)

data返回以下内容:[dput(data)]

data <- structure(c(NA, NA, NA, NA, NA, NA, "AMERICAN CAR & FDRY CO", 
"AMERICAN CAR & FDRY CO", "AMERICAN CAR & FDRY CO", "ALASKA JUNEAU GOLD MNG CO", 
"AMERICAN CAR & FDRY CO", "ALASKA JUNEAU GOLD MNG CO", "A", "A", 
"A", "A", "A", "A", "109.0000", "109.1250", "111.0000", "  1.8750", 
"110.5000", "  1.7500", "C", "0.001147", "0.017182", "C", "-0.004505", 
"-0.066667", "C", "0.001147", "0.017182", "C", "-0.004505", "-0.066667", 
NA, " 0.005893", " 0.001277", " 0.001277", "-0.003984", "-0.003984", 
NA, " 0.009516", " 0.005780", " 0.005780", "-0.001927", "-0.001927", 
NA, NA, NA, NA, NA, NA), .indexCLASS = "Date", tclass = "Date", .indexTZ = "UTC", tzone = "UTC",    
class = c("xts", 
"zoo"), index = structure(c(-1388620800, -1388448000, -1388275200, 
-1388275200, -1388188800, -1388188800), tzone = "UTC", tclass = "Date"), .Dim = c(6L, 
9L), .Dimnames = list(NULL, c("TICKER", "COMNAM", "TRDSTAT", 
"PRC", "RET", "RETX", "vwretd", "ewretd", "sprtrn")))

可重复数据

crsp <- structure(list(PERMNO = c(10006L, 10006L, 10006L, 10006L, 10006L, 
10006L, 10006L, 10006L, 10006L, 10014L, 10014L, 10014L, 10014L, 
10014L, 10014L, 10014L, 10014L), date = c(19251231L, 19260102L, 
19260104L, 19260105L, 19260106L, 19260107L, 19301229L, 19301230L, 
19301231L, 19260104L, 19260105L, 19260106L, 19260107L, 19260108L, 
19260109L, 19260111L, 19260112L), TICKER = c(NA_integer_, NA_integer_, 
NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_, 
NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_, 
NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_
), COMNAM = c("AMERICAN CAR & FDRY CO", "AMERICAN CAR & FDRY CO", 
"AMERICAN CAR & FDRY CO", "AMERICAN CAR & FDRY CO", "AMERICAN CAR & FDRY CO", 
"AMERICAN CAR & FDRY CO", "AMERICAN CAR & FDRY CO", "AMERICAN CAR & FDRY CO", 
"AMERICAN CAR & FDRY CO", "ALASKA JUNEAU GOLD MNG CO", "ALASKA JUNEAU GOLD MNG CO", 
"ALASKA JUNEAU GOLD MNG CO", "ALASKA JUNEAU GOLD MNG CO", "ALASKA JUNEAU GOLD MNG CO", 
"ALASKA JUNEAU GOLD MNG CO", "ALASKA JUNEAU GOLD MNG CO", "ALASKA JUNEAU GOLD MNG CO"
), TRDSTAT = c("A", "A", "A", "A", "A", "A", "A", "A", "A", "A", 
"A", "A", "A", "A", "A", "A", "A"), PRC = c(109, 109.125, 111, 
110.5, 110.5, 110.5, 24.25, 24.5, 27.5, 1.875, 1.75, 1.75, -1.8125, 
1.75, -1.6875, 1.625, NA), RET = c("C", "0.001147", "0.017182", 
"-0.004505", "0.000000", "0.000000", "-0.034826", "0.010309", 
"0.122449", "C", "-0.066667", "0.000000", "0.035714", "-0.034483", 
"-0.035714", "-0.037037", ""), RETX = c("C", "0.001147", "0.017182", 
"-0.004505", "0.000000", "0.000000", "-0.034826", "0.010309", 
"0.122449", "C", "-0.066667", "0.000000", "0.035714", "-0.034483", 
"-0.035714", "-0.037037", ""), vwretd = c(NA, 0.005893, 0.001277, 
-0.003984, -0.000172, 0.007211, -0.002367, 0.020064, 0.016505, 
0.001277, -0.003984, -0.000172, 0.007211, -0.000804, 0.003384, 
-0.00868, -0.001027), ewretd = c(NA, 0.009516, 0.00578, -0.001927, 
0.001182, 0.008453, -0.017949, 0.016016, 0.052093, 0.00578, -0.001927, 
0.001182, 0.008453, -0.001689, 0.003312, -0.009943, -0.003623
), sprtrn = c(NA_integer_, NA_integer_, NA_integer_, NA_integer_, 
NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_, 
NA_integer_, NA_integer_, NA_integer_, NA_integer_, NA_integer_, 
NA_integer_, NA_integer_, NA_integer_)), .Names = c("PERMNO", 
"date", "TICKER", "COMNAM", "TRDSTAT", "PRC", "RET", "RETX", 
"vwretd", "ewretd", "sprtrn"), row.names = c(1L, 2L, 3L, 4L, 
5L, 6L, 1483L, 1484L, 1485L, 1486L, 1487L, 1488L, 1489L, 1490L, 
1491L, 1492L, 1493L), class = "data.frame")

格式date列(与上面相同的代码)

crsp$DATE <- as.Date(as.character(crsp$date),format="%Y%m%d")
data <- xts(crsp[,3:11], crsp$DATE)

更新

@akrun生成的代码可以有效地处理样本数据,但我遇到了实际数据集的问题:

# this code works well on actual data
crsp1 <- crsp[,c(4, 6:12)];library(reshape2)

# this is the part where I am having difficulty with
crsp2 <- dcast(crsp1[,c(1:2, 8)], DATE~COMNAM, value.var="PRC")

它返回以下错误:Aggregation function missing: defaulting to length 这是crsp2返回的示例:

crsp2 <- structure(list(DATE = structure(c(-16072, -16070, -16068, -16067, 
-16066), class = "Date"), `A P W PAPER CO` = c(0, 0, 0, 0, 0), 
`ABITIBI POWER & PAPER LTD` = c(1, 1, 1, 1, 1), `ABRAHAM & STRAUSS INC` = c(0, 
0, 0, 0, 0), `ADAMS EXPRESS CO` = c(1, 1, 1, 1, 1)), .Names = c("DATE", 
"A P W PAPER CO", "ABITIBI POWER & PAPER LTD", "ABRAHAM & STRAUSS INC", 
"ADAMS EXPRESS CO"), row.names = c(NA, 5L), class = "data.frame")

format而言,它会返回所需的输出,但问题是它会返回1&0;而0而不是实际值... 我相信它会返回找到/开始数据的1,以及没有找到数据的0&#s。我后来尝试指定Aggregation function

# Specification of aggregation function plus "fill" to try to fix length errors & "drop" to
# avoid values being dropped
crsp2 <- dcast(crsp1[,c(1:2, 8)], DATE ~ COMNAM, value.var="RET", fun.aggregate= length, fill=  
0);library(plyr);

这一次,我没有得到任何错误,但仍然得到1和0。也许最好按COMNAM分隔数据,然后按DATE列进行索引,以便我可以merge()cbind()获得所需的输出。但我不知道是否有更好的方法?我也尝试了以下功能,但让它们运行了6个多小时后又回来了,哪里还在运行...似乎无法使它们正常工作......

# daply() from library(plyr)
crsp2 <- daply(crsp1[,c(1:2, 8)], .(DATE,COMNAM), function(x) x$PRC, .parallel=TRUE)


# reshape()
crsp2 <- reshape(crsp1[,c(1:2, 8)], idvar="COMNAM", timevar="DATE", direction="wide") 

1 个答案:

答案 0 :(得分:2)

这里有两个与xts相关的问题:

  1. xts假设索引的弱单调排序。允许重复的索引值,但您不应该依赖重复值的排序(如果您有重复的索引条目,则可能会抛出大量警告)。因此,对于时间戳数据,我们使用make.time.unique()之类的帮助器,它们添加最小的epsilon来分隔它们。

  2. xts使用矩阵作为数据容器,这意味着您不能混合类型(例如,您只能拥有数字数据,或只能使用字符数据等)。

  3. 所以在这里,添加data.frame列或Date的{​​{1}}可能会更好。