Question

我的数据包含两列（可能更多）标识符（通常是长字符串）。这些有时会有所不同，输入错误或随时间变化。我想识别数据中的唯一主题。这需要识别通过案例ID在某种程度上相关联的案例组。

一个例子

df <- data.frame(ida = c("A", "B", "C", "C", "D", "E"),
                 idb = c(1, 1, 3, 4, 4, 7),
                 trueid = c("id1", "id1", "id2", "id2", "id2", "id3"))

> df
  ida idb trueid
1   A   1    id1
2   B   1    id1
3   C   3    id2
4   C   4    id2
5   D   4    id2
6   E   7    id3

id1的标识符是"A", "B", 1，id2 "C", "D", 3, 4和id3 "E", 7的标识符。

我不知道trueid，但需要使用ida和idb列中的信息来找到它。

该解决方案需要扩展到具有数万个唯一ID的数百万个观察值。我已经在使用data.table。

扩展：在另一种情况下，有两列以上，有些列可能对其他列有帮助，即具有相同的标识符。我不知道哪些列对哪些内容有用。我认为类型可以忽略，所有列都是字符串，也可以安全地转换。

另一个示例：

df <- data.frame(ida = c("A", "B", "C", "C", "D", "E"),
                 idb = c("1", "2", "3", "4", "4", "7"),
                 idc = c("1", "1", "2", "3", "4", "5"),
                 idd = c("1", "A", "2", "3", "4", "5"),
                 trueid = c("id1", "id1", "id1", "id1", "id1", "id2"))

> df
  ida idb idc idd trueid
1   A   1   1   1    id1
2   B   2   1   A    id1
3   C   3   2   2    id1
4   C   4   3   3    id1
5   D   4   4   4    id1
6   E   7   5   5    id2

编辑：正如评论者所指出的，这本质上是在图中找到完整子图的问题。阅读更多内容后，我知道可以使用library(igraph)解决此问题。我将问题悬而未决，因为我希望依靠base，data.table或dplyr的解决方案。我无法在正在使用的服务器上轻松安装软件包，安装igraph涉及很多繁琐的工作和延迟。

Edit2 ：对于阅读此书并面临类似问题的任何人：zx8754使用igraph的答案在具有更多组的较大（模拟）数据上的速度要快得多（几个数量级）。如果您有机会使用igraph，请这样做。

Answer 1

使用 igraph ：

# example input, I removed "trueid" column
df <- data.frame(ida = c("A", "B", "C", "C", "D", "E"),
                 idb = c("1", "2", "3", "4", "4", "7"),
                 idc = c("1", "1", "2", "3", "4", "5"),
                 idd = c("1", "A", "2", "3", "4", "5"))
#trueid = c("id1", "id1", "id1", "id1", "id1", "id2")

library(igraph)

# set up connections
# Improved version suggested by @thelatemail in the comments
x <- cbind(df[ 1 ], unlist(df[ -1 ]))

# original clumsy version (do not use)
# x <- unique(do.call(rbind, lapply(1:(ncol(df) - 1), function(i) setNames(df[, c(i, i + 1) ], c("from", "to")))))

# convert to graph object
g <- graph_from_data_frame(x)        

# plot if you wish to visualise
plot(g)

# this is the solution, add membership ids to original input dataframe
merge(df, data.frame(grp = clusters(g)$membership),
      by.x = "ida", by.y = 0)
#   ida idb idc idd grp
# 1   A   1   1   1   1
# 2   B   2   1   A   1
# 3   C   3   2   2   1
# 4   C   4   3   3   1
# 5   D   4   4   4   1
# 6   E   7   5   5   2

Answer 2

这是使用data.table的递归方法：

#convert into a long format for easier processing
mDT <- melt(DT[, rn := .I], id.var="rn", variable.name="V", value.name="ID")[,
    tid := NA_integer_]

#the recursive function
link <- function(ids, label) {
    #identify the rows in DT containing ids and extract the IDs
    newids <- mDT[mDT[.(ID=ids), on=.(ID), .(rn=rn)], on=.(rn), allow.cartesian=TRUE,
        unique(ID)]

    #update those rows to the same group
    mDT[mDT[.(ID=ids), on=.(ID), .(rn=rn)], on=.(rn), tid := label]

    if (length(setdiff(newids, ids)) > 0L) {
        #call the recursive function if there are new ids
        link(newids, label)
    }
}

#get the first id that is not labelled yet
id <- mDT[is.na(tid), ID[1L]]
grp <- 1L
while(!is.na(id)) {
    #use recursive function to link them up
    link(id, grp)

    #repeat for next id that is not part of any group yet
    id <- mDT[is.na(tid), ID[1L]]
    grp <- grp + 1L
}

#update original DT with tid
DT[mDT, on=.(rn), tid := tid]

数据：

library(data.table)
DT <- data.table(ida = c("A", "B", "C", "C", "D", "E"),
    idb = c("1", "2", "3", "4", "4", "7"),
    idc = c("1", "1", "2", "3", "4", "5"),
    idd = c("1", "A", "2", "3", "4", "5"))

在几列中查找唯一的一组标识符/组

2 个答案: