Question

我有两个大型数据集，一个大约五十万个记录，另一个大约70K。这些数据集有地址。如果较小数据集中的任何地址存在于大数据集中，我想匹配。正如您所想象的那样，地址可以用不同的方式和不同的情况/拼写等来写。除了这个地址可以复制，如果只写到建筑物级别。所以不同的公寓有相同的地址。我做了一些研究并找出了可以使用的包stringdist。

我做了一些工作并设法根据距离获得最接近的匹配。但是，我无法返回地址匹配的相应列。

下面是一个示例虚拟数据以及我为解释情况而创建的代码

library(stringdist)
Address1 <- c("786, GALI NO 5, XYZ","rambo, 45, strret 4, atlast, pqr","23/4, 23RD FLOOR, STREET 2, ABC-E, PQR","45-B, GALI NO5, XYZ","HECTIC, 99 STREET, PQR","786, GALI NO 5, XYZ","rambo, 45, strret 4, atlast, pqr")
Year1 <- c(2001:2007)

Address2 <- c("abc, pqr, xyz","786, GALI NO 4 XYZ","45B, GALI NO 5, XYZ","del, 546, strret2, towards east, pqr","23/4, STREET 2, PQR","abc, pqr, xyz","786, GALI NO 4 XYZ","45B, GALI NO 5, XYZ","del, 546, strret2, towards east, pqr","23/4, STREET 2, PQR")
Year2 <- c(2001:2010)

df1 <- data.table(Address1,Year1)
df2 <- data.table(Address2,Year2)
df2[,unique_id := sprintf("%06d", 1:nrow(df2))]

fn_match = function(str, strVec, n){
  strVec[amatch(str, strVec, method = "dl", maxDist=n,useBytes = T)]
}

df1[!is.na(Address1)
    , address_match := 
      fn_match(Address1, df2$Address2,3)
    ]

这将返回基于距离3的闭合字符串匹配，但是我还希望列有＆＃34;年＆＃34;和＆＃34; unique_id＆＃34;来自df1中的df2。这将帮助我知道字符串与df2匹配的数据行。所以最后我想知道 df1中的每一行 根据指定的距离，df2的壁橱匹配是什么，并且匹配的行具有特定的 ＆＃34;年＆＃34; 和＆＃34; unique_id＆＃34;来自df2 。

我想与merge（左连接）有关，但我不知道如何合并保留重复项并确保我有与df1（小数据集）相同的行数。

任何类型的解决方案都会有所帮助!!

Answer 1

你有90％的路在那里......

你说你想要

知道字符串与df2匹配的数据行

您只需了解您已有的代码即可。见?amatch：

amatch返回x中table最接近匹配的位置。当存在具有相同最小距离度量的多个匹配时，返回第一个匹配。

换句话说，amatch为df2（table中与df1中每个地址最匹配的行中的行提供了索引（这是你的x）。您通过返回新地址来过早地包装此索引。

相反，检索索引本身以查找或 unique_id（如果您确信它确实是唯一ID），则为左连接。

两种方法的说明：

library(data.table) # you forgot this in your example
library(stringdist)
df1 <- data.table(Address1 = c("786, GALI NO 5, XYZ","rambo, 45, strret 4, atlast, pqr","23/4, 23RD FLOOR, STREET 2, ABC-E, PQR","45-B, GALI NO5, XYZ","HECTIC, 99 STREET, PQR","786, GALI NO 5, XYZ","rambo, 45, strret 4, atlast, pqr"),
                  Year1 = 2001:2007) # already a vector, no need to combine
df2 <- data.table(Address2=c("abc, pqr, xyz","786, GALI NO 4 XYZ","45B, GALI NO 5, XYZ","del, 546, strret2, towards east, pqr","23/4, STREET 2, PQR","abc, pqr, xyz","786, GALI NO 4 XYZ","45B, GALI NO 5, XYZ","del, 546, strret2, towards east, pqr","23/4, STREET 2, PQR"),
                  Year2=2001:2010)
df2[,unique_id := sprintf("%06d", .I)] # use .I, it's neater

# Return position from strVec of closest match to str
match_pos = function(str, strVec, n){
  amatch(str, strVec, method = "dl", maxDist=n,useBytes = T) # are you sure you want useBytes = TRUE?
}

# Option 1: use unique_id as a key for left join
df1[!is.na(Address1) | nchar(Address1>0), # I would exclude only on NA_character_ but also empty string, perhaps string of length < 3
    unique_id := df2$unique_id[match_pos(Address1, df2$Address2,3)] ]
merge(df1, df2, by='unique_id', all.x=TRUE) # see ?merge for more options

# Option 2: use the row index
df1[!is.na(Address1) | nchar(Address1>0),
    df2_pos := match_pos(Address1, df2$Address2,3) ] 
df1[!is.na(df2_pos), (c('Address2','Year2','UniqueID')):=df2[df2_pos,.(Address2,Year2,unique_id)] ][]

Answer 2

以下是使用fuzzyjoin包的解决方案。它使用dplyr - 类似语法和stringdist作为模糊匹配的可能类型之一。

您可以使用stringdist method =＆＃34; dl＆＃34; （或其他可能更好的工作）。

为了满足您的要求＆＃34;确保我的行数与df1＆＃34;相同，我使用了一个较大的max_dist，然后使用dplyr::group_by和dplyr::top_n来获取与最小距离最佳匹配。这是fuzzyjoin的开发人员dgrtwo suggested。（希望它将来会成为软件包本身的一部分。）

（我还必须假设在距离关系的情况下采用最大年份2。）

<强>代码：

library(data.table, quietly = TRUE)
df1 <- data.table(Address1 = c("786, GALI NO 5, XYZ","rambo, 45, strret 4, atlast, pqr","23/4, 23RD FLOOR, STREET 2, ABC-E, PQR","45-B, GALI NO5, XYZ","HECTIC, 99 STREET, PQR","786, GALI NO 5, XYZ","rambo, 45, strret 4, atlast, pqr"),
                  Year1 = 2001:2007) 
df2 <- data.table(Address2=c("abc, pqr, xyz","786, GALI NO 4 XYZ","45B, GALI NO 5, XYZ","del, 546, strret2, towards east, pqr","23/4, STREET 2, PQR","abc, pqr, xyz","786, GALI NO 4 XYZ","45B, GALI NO 5, XYZ","del, 546, strret2, towards east, pqr","23/4, STREET 2, PQR"),
                  Year2=2001:2010)
df2[,unique_id := sprintf("%06d", .I)]

library(fuzzyjoin, quietly = TRUE); library(dplyr, quietly = TRUE)
stringdist_join(df1, df2, 
                by = c("Address1" = "Address2"), 
                mode = "left", 
                method = "dl", 
                max_dist = 99, 
                distance_col = "dist") %>%
  group_by(Address1, Year1) %>%
  top_n(1, -dist) %>%
  top_n(1, Year2)

<强>结果：

# A tibble: 7 x 6
# Groups:   Address1, Year1 [7]
                                Address1 Year1                             Address2 Year2 unique_id  dist
                                   <chr> <int>                                <chr> <int>     <chr> <dbl>
1                    786, GALI NO 5, XYZ  2001                   786, GALI NO 4 XYZ  2007    000007     2
2       rambo, 45, strret 4, atlast, pqr  2002 del, 546, strret2, towards east, pqr  2009    000009    17
3 23/4, 23RD FLOOR, STREET 2, ABC-E, PQR  2003                  23/4, STREET 2, PQR  2010    000010    19
4                    45-B, GALI NO5, XYZ  2004                  45B, GALI NO 5, XYZ  2008    000008     2
5                 HECTIC, 99 STREET, PQR  2005                  23/4, STREET 2, PQR  2010    000010    11
6                    786, GALI NO 5, XYZ  2006                   786, GALI NO 4 XYZ  2007    000007     2
7       rambo, 45, strret 4, atlast, pqr  2007 del, 546, strret2, towards east, pqr  2009    000009    17

R模糊字符串匹配，根据匹配的字符串

2 个答案: