在音像中分裂说话者和对话

时间:2016-12-12 12:02:57

标签: r text-mining

我有以下文件:

  总统诺伯特·拉姆特博士:我宣布会议开幕。

     

我现在请Bundesminister Alexander Dobrindt发言。

     

(CDU / CSU的掌声和社民党的代表)

     

运输和数字基础设施部长Alexander Dobrindt:

     

女士们,先生们。我们今天将开始对有史以来最大的基础设施投资,超过2700亿欧元,超过1000个项目和明确的融资视角。

     

(Volker Kauder [CDU / CSU]:Genau!)

     

(CDU / CSU和SPD的掌声)

当我阅读那些.txt文档时,我想创建一个第二列,表明发言人姓名。

所以我尝试的是首先创建一个包含所有可能名称的列表并替换它们。

library(qdap)

members <- c("Alexander Dobrindt, Minister for Transport and Digital Infrastructure:","President Dr. Norbert Lammert:")
members_r <- c("@Alexander Dobrindt, Minister for Transport and Digital Infrastructure:","@President Dr. Norbert Lammert:")

prok <- scan(".txt", what = "character", sep = "\n")
prok <- mgsub(members,members_r,prok)

prok <- as.data.frame(prok)
prok$speaker <- grepl("@[^\\@:]*:",prok$prok, ignore.case = T)

我的计划是获取@和之间的名称:通过正则表达式,如果说话者== true并向下应用它,直到有一个不同的名称(并明显删除所有的掌声/喊叫括号),但这也是我的位置不知道我怎么能这样做。

3 个答案:

答案 0 :(得分:2)

这是方法:

      require (qdap)
      #text is the document text

      # remove round brackets and text b/w ()
      a <- bracketX(text, "round") 

      names <- c("President Dr. Norbert Lammert","Alexander Dobrindt" )
      searchString <- paste(names[1],names[2], sep = ".+")

      # Get string from names[1] till names[2] with the help of searchString
      string <- regmatches(a, regexpr(searchString, a))

      # remove names[2] from string
      string <- gsub(names[2],"",string)

当代码超过2个

时,可以循环使用此代码

答案 1 :(得分:1)

这似乎有效

library(qdap)

members <- c("Alexander Dobrindt, Minister for Transport and Digital Infrastructure:","President Dr. Norbert Lammert:")
members_r <- c("@Alexander Dobrindt, Minister for Transport and Digital Infrastructure:","@President Dr. Norbert Lammert:")

testprok <- read.table("txt",header=FALSE,quote = "\"",comment.char="",sep="\t")

testprok$V1 <- mgsub(members,members_r,testprok$V1)

testprok$V2 <- ifelse(grepl("@[^\\@:]*:",testprok$V1),testprok$V1,NA)       

####function from http://stackoverflow.com/questions/7735647/replacing-nas-with-latest-non-na-value         
repeat.before = function(x) {   # repeats the last non NA value. Keeps leading NA
  ind = which(!is.na(x))      # get positions of nonmissing values
  if(is.na(x[1]))             # if it begins with a missing, add the 
    ind = c(1,ind)        # first position to the indices
  rep(x[ind], times = diff(   # repeat the values at these indices
    c(ind, length(x) + 1) )) # diffing the indices + length yields how often 
}                               # they need to be repeated

testprok$V2 = repeat.before(testprok$V2)

答案 2 :(得分:1)

这是一种严重依赖dplyr的方法。

首先,我在示例文本中添加了一个句子,以说明为什么我们不能使用冒号来识别说话人姓名。

sampleText <-
"President Dr. Norbert Lammert: I declare the session open.

I will now give the floor to Bundesminister Alexander Dobrindt.

(Applause of CDU/CSU and delegates of the SPD)

Alexander Dobrindt, Minister for Transport and Digital Infrastructure:

Ladies and Gentleman. We will today start the biggest investment in infrastructure that ever existed, with over 270 billion Euro, over 1 000 projects and a clear financing perspective.

(Volker Kauder [CDU/CSU]: Genau!)

(Applause of the CDU/CSU and the SPD)

This sentence right here: it is an example of a problem"

然后我将文本拆分为模拟您正在阅读的格式(这也将每个语音放在列表的一部分中)。

splitText <- strsplit(sampleText, "\n")

然后,我将所有潜在的发言者(结肠前的任何内容)拉出

allSpeakers <- lapply(splitText, function(thisText){
  grep(":", thisText, value = TRUE) %>%
    gsub(":.*", "", .) %>%
    gsub("\\(", "", .)
}) %>%
  unlist() %>%
  unique()

这给了我们:

[1] "President Dr. Norbert Lammert"                                        
[2] "Alexander Dobrindt, Minister for Transport and Digital Infrastructure"
[3] "Volker Kauder [CDU/CSU]"                                              
[4] "This sentence right here" 

显然,最后一个不是合法的名称,因此应从我们的发言人名单中排除:

legitSpeakers <-
  allSpeakers[-4]

现在,我们已准备好完成演讲。我在下面列出了逐步注释,而不是在文本中进行描述

speechText <- lapply(splitText, function(thisText){

  # Remove applause and interjections (things in parentheses)
  # along with any blank lines; though you could leave blanks if you want
  cleanText <-
    grep("(^\\(.*\\)$)|(^$)", thisText
         , value = TRUE, invert = TRUE)

  # Split each line by a semicolor
  strsplit(cleanText, ":") %>%
    lapply(function(x){
      # Check if the first element is a legit speaker
      if(x[1] %in% legitSpeakers){
        # If so, set the speaker, and put the statement in a separate portion
        # taking care to re-collapse any breaks caused by additional colons
        out <- data.frame(speaker = x[1]
                          , text = paste(x[-1], collapse = ":"))
      } else{
        # If not a legit speaker, set speaker to NA and reset text as above
        out <- data.frame(speaker = NA
                          , text = paste(x, collapse = ":"))
      }
      # Return whichever version we made above
      return(out)
    }) %>%
    # Bind all of the rows together
    bind_rows %>%
    # Identify clusters of speech that go with a single speaker
    mutate(speakingGroup = cumsum(!is.na(speaker))) %>%
    # Group by those clusters
    group_by(speakingGroup) %>%
    # Collapse that speaking down into a single row
    summarise(speaker = speaker[1]
              , fullText = paste(text, collapse = "\n"))
})

这会产生

[[1]]

speakingGroup  speaker                                                                fullText                                                                                                                                                                                                                                        

            1  President Dr. Norbert Lammert                                          I declare the session open.\nI will now give the floor to Bundesminister Alexander Dobrindt.                                                                                                                                                     
            2  Alexander Dobrindt, Minister for Transport and Digital Infrastructure  Ladies and Gentleman. We will today start the biggest investment in infrastructure that ever existed, with over 270 billion Euro, over 1 000 projects and a clear financing perspective.\nThis sentence right here: it is an example of a problem

如果您希望单独使用每行文字,请将最后的summarise替换为mutate(speaker = speaker[1]),并为每一行语音添加一行,如下所示:

speaker                                                                text                                                                                                                                                                                      speakingGroup
President Dr. Norbert Lammert                                          I declare the session open.                                                                                                                                                                           1
President Dr. Norbert Lammert                                          I will now give the floor to Bundesminister Alexander Dobrindt.                                                                                                                                       1
Alexander Dobrindt, Minister for Transport and Digital Infrastructure                                                                                                                                                                                                        2
Alexander Dobrindt, Minister for Transport and Digital Infrastructure  Ladies and Gentleman. We will today start the biggest investment in infrastructure that ever existed, with over 270 billion Euro, over 1 000 projects and a clear financing perspective.              2
Alexander Dobrindt, Minister for Transport and Digital Infrastructure  This sentence right here: it is an example of a problem                                                                                                                                               2