Question

我使用ff和ffbase个软件包来组合两个ffdf个对象，但是当我使用merge语句时，它会从100个行开始目标ffdf到800万行。

ffdf1是1百万行乘6列：

> summary(ffdf2)
              Length  Class     Mode
userid        1000000 ff_vector list
V2            1000000 ff_vector list
V3            1000000 ff_vector list
V4            1000000 ff_vector list
V5            1000000 ff_vector list
V6            1000000 ff_vector list

ffdf2是大约2000万行X 3列，如下所示：

  userid      gender      age
    1         1          3
    2         1          2
    3         2          5
    4         0          4
    5         2          3
    ...       ...       ...

我使用以下代码合并两者：

ffdf3 <- merge(ffdf1, ffdf2, by.x="userid",by.y="userid",all.x=T,sort=F)

结果如下：

> summary(ffdf3)
                  Length  Class     Mode
    userid        8000000 ff_vector list
    V2            8000000 ff_vector list
    V3            8000000 ff_vector list
    V4            8000000 ff_vector list
    V5            8000000 ff_vector list
    V6            8000000 ff_vector list
    gender        8000000 ff_vector list
    age           8000000 ff_vector list

为什么长度从1毫米到8毫米的任何想法？

编辑：

当我尝试这个时：

ffdf3 <- merge(ffdf1, ffdf2, by.x="userid",by.y="userid",all.x=F,sort=F)

我明白了：

> summary(ffdf3)
                      Length  Class     Mode
        userid        740383 ff_vector list
        V2            740383 ff_vector list
        V3            740383 ff_vector list
        V4            740383 ff_vector list
        V5            740383 ff_vector list
        V6            740383 ff_vector list
        gender        740383 ff_vector list
        age           740383 ff_vector list

此处还有运行合并的输出：

2012-05-13 14:49:06, x has 2 chunks, y has 8 chunks
2012-05-13 14:49:06, working on x chunk 1:500000
2012-05-13 14:49:07, working on y chunk 1:2958661
2012-05-13 14:49:16, working on y chunk 2958662:5917322
2012-05-13 14:49:32, working on y chunk 5917323:8875983
2012-05-13 14:49:45, working on y chunk 8875984:11834644
2012-05-13 14:49:57, working on y chunk 11834645:14793305
2012-05-13 14:50:09, working on y chunk 14793306:17751966
2012-05-13 14:50:20, working on y chunk 17751967:20710627
2012-05-13 14:50:30, working on y chunk 20710628:23669283
2012-05-13 14:50:40, working on x chunk 500001:1000000
2012-05-13 14:50:41, working on y chunk 1:2958661
2012-05-13 14:50:52, working on y chunk 2958662:5917322
2012-05-13 14:51:03, working on y chunk 5917323:8875983
2012-05-13 14:51:14, working on y chunk 8875984:11834644
2012-05-13 14:51:24, working on y chunk 11834645:14793305
2012-05-13 14:51:36, working on y chunk 14793306:17751966
2012-05-13 14:51:47, working on y chunk 17751967:20710627
2012-05-13 14:51:58, working on y chunk 20710628:23669283

同样ffdf1包含677840个唯一userid个。因此，1毫米行中有一些重复。

Answer 1

merge.ffdf包含一个错误，目前只允许正确进行内连接，而不是all.x = TRUE和all.y = FALSE。该函数在@ http://code.google.com/p/fffunctions/上运行。问题是，当您没有匹配的记录时，在进行左外连接时，需要更改vmodes才能正确地允许NA。这项工作正在进行中。

FYI。这已在http://code.google.com/p/fffunctions/的开发版本中解决，并将在未来几周内上传到CRAN。

R ff / ffbase合并创建行

1 个答案: