Question

我有一个这样的清单：

.NET
ABAP
Access
Account Management
Accounting
Active Directory
Agile Methodologies
Agile Project Management
AJAX
Algorithms
Analysis
Android
Android Development
AngularJS
Ant
Apache
ASP
ASP.NET
B2B
Banking
BPMN
Budgets
Business Analysis
Business Development
Business Intelligence
Business Planning
Business Process
Business Process Design
Business Process...

我想减少我必须分析的变量数量，以创建更抽象的类别。从上一个列表中，每个单词都是我的一个变量。

我发现有word2vec，但我无法找到CRAN文档。

我该如何使用它？我该怎么处理这些数据？

Answer 1

不是word2vec，而是另外看this post：

library(XML)
library(dplyr)
library(RecordLinkage)
df <- data.frame(words=capture.output(htmlParse("https://stackoverflow.com/questions/35904182/word2vec-for-text-mining-categories")[["//div/pre/code/text()"]]))
df %>% compare.dedup(strcmp = TRUE) %>%
             epiWeights() %>%
             epiClassify(0.8) %>%
             getPairs(show = "links", single.rows = TRUE) -> matches
left_join(mutate(df,ID = 1:nrow(df)), 
          select(matches,id1,id2) %>% arrange(id1) %>% filter(!duplicated(id2)), 
          by=c("ID"="id2")) %>%
    mutate(ID = ifelse(is.na(id1), ID, id1) ) %>%
    select(-id1) -> dfnew
head(dfnew, 30)
#                       words ID
# 1                      .NET  1
# 2                      ABAP  2
# 3                    Access  3
# 4        Account Management  4 # <--
# 5                Accounting  4 # <--
# 6          Active Directory  6
# 7       Agile Methodologies  7 # <--
# 8  Agile Project Management  7 # <--
# 9                      AJAX  9
# 10               Algorithms 10
# 11                 Analysis 11
# 12                  Android 12 # <--
# 13      Android Development 12 # <--
# 14                AngularJS 14
# 15                      Ant 15
# 16                   Apache 16
# 17                      ASP 17 # <--
# 18                  ASP.NET 17 # <--
# 19                      B2B 19
# 20                  Banking 20
# 21                     BPMN 21
# 22                  Budgets 22
# 23        Business Analysis 23 # <--
# 24     Business Development 23 # <--
# 25    Business Intelligence 23 # <--
# 26        Business Planning 23 # <--
# 27         Business Process 23 # <--
# 28  Business Process Design 23 # <--
# 29      Business Process... 23 # <--
# 30        Business Strategy 23 # <--

dfnew$ID可能是基于jaro-winkler字符串距离的抽象类别。可能需要对您的真实数据进行一些微调。

word2vec用于文本挖掘类别

1 个答案: