从作者关系中提取国家名称

时间:2011-03-15 21:06:32

标签: r text nlp

我目前正在探索从作者联盟(PubMed文章)中提取国家名称的可能性,我的样本数据如下:

Mechanical and Production Engineering Department, National University of Singapore.

Cancer Research Campaign Mammalian Cell DNA Repair Group, Department of Zoology, Cambridge, U.K.

Cancer Research Campaign Mammalian Cell DNA Repair Group, Department of Zoology, Cambridge, UK.

Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, IN 46285.

最初我尝试删除标点符号并将矢量拆分为单词,然后将其与维基百科的国家/地区名称列表进行比较,但我没有成功。

任何人都可以建议我一个更好的方法吗?我更喜欢R中的解决方案,因为我必须进一步分析并在R中生成图形。

3 个答案:

答案 0 :(得分:7)

这是一个简单的解决方案,可能会让你开始一些方式。它使用包含地图包中的城市和国家数据的数据库。如果你能掌握一个更好的数据库,修改代码应该很简单。

library(maps)
library(plyr)

# Load data from package maps
data(world.cities)

# Create test data
aa <- c(
    "Mechanical and Production Engineering Department, National University of Singapore.",
    "Cancer Research Campaign Mammalian Cell DNA Repair Group, Department of Zoology, Cambridge, U.K.",
    "Cancer Research Campaign Mammalian Cell DNA Repair Group, Department of Zoology, Cambridge, UK.",
    "Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, IN 46285."
)

# Remove punctuation from data
caa <- gsub(aa, "[[:punct:]]", "")    ### *Edit*

# Split data at word boundaries
saa <- strsplit(caa, " ")

# Match on cities in world.cities
# Assumes that if multiple matches, the last takes precedence, i.e. max()
llply(saa, function(x)x[max(which(x %in% world.cities$name))])

# Match on country in world.countries
llply(saa, function(x)x[which(x %in% world.cities$country.etc)])

这是城市的结果:

[[1]]
[1] "Singapore"

[[2]]
[1] "Cambridge"

[[3]]
[1] "Cambridge"

[[4]]
[1] "Indianapolis"

各国的结果:

[[1]]
[1] "Singapore"

[[2]]
[1] "UK"

[[3]]
[1] "UK"

[[4]]
character(0)

通过一些数据清理,您可以对此做些什么。

答案 1 :(得分:1)

一种方法是拆分字符串以隔离地理信息(例如删除第一个昏迷的所有内容),然后将结果提交给地理编码服务。

例如,Google地理编码API允许发送地址并获取本地化和相应的地理信息,例如国家/地区。我不认为有一个现成的R包来做,但你可以在这里找到一些功能,例如:

Geocoding in R with Google Maps

还有其他语言的扩展,例如Ruby:

http://geokit.rubyforge.org/

如果我没记错的话,这还取决于您的观察次数,例如,免费的Google API限制为大约200个地址/ IP /天。

答案 2 :(得分:0)

@Andrie的回答很好,但是它错过了多个单词(例如,新西兰或纽约。第二个例子令人担忧,因为它将被标记为与英国约克而不是美国纽约的比赛。

这种选择应该可以更好地捕获这些情况。

library(maps)
library(plyr)

# Load data from package maps
data(world.cities)

# Create test data
aa <- c(
    "Mechanical and Production Engineering Department, National University of Singapore.",
    "Cancer Research Campaign Mammalian Cell DNA Repair Group, Department of Zoology, Cambridge, U.K.",
    "Cancer Research Campaign Mammalian Cell DNA Repair Group, Department of Zoology, Cambridge, UK.",
    "Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, IN 46285."
)

saa <- sapply(aa, strsplit, split = ", ", USE.NAMES = FALSE)
llply(saa, function(x)x[which(x %in% world.cities$name)])
llply(saa, function(x)x[which(x %in% world.cities$country.etc)])

缺点是任何没有特定国家或城市字段的条目都不会返回任何内容,例如以新加坡大学为例。

城市:

[[1]]
character(0)

[[2]]
[1] "Cambridge"

[[3]]
[1] "Cambridge"

[[4]]
[1] "Indianapolis"

对我来说,这比多词城市/国家问题少。选择最适合您的数据的那个。也许有一种将两者结合的方法?