带有20万条记录的数据帧的内存问题

时间:2019-06-27 14:26:09

标签: r bigdata ff r-bigmemory

我有2个矩阵,每个矩阵有20万条记录(一个是很大的get_sentences-review_phrases,另一个是review_scores)。 将它们绑定到数据帧,需要将其写入csv,但会出现内存错误。我该怎么办?软件包bigmemory或ff有帮助吗?

我正在使用Windows 10 64位8GB RAM。内存限制给出以下输出:

memory.limit()
[1] 1.759219e+13
memory.size()
[1] 418.85

代码如下:

  sentiment_matrix_final = data.frame (Comment = review_phrases)

 df_scores = data.frame(Scores = review_scores)

 sentiment_matrix_final = cbind(sentiment_matrix_final, df_scores)

 sentiment_matrix_final = data.frame (Comment = review_phrases, Score = review_scores)

 str(sentiment_matrix_final)


  # Write CSV in R
  write.csv(sentiment_matrix_final, file = "Score_sentiment_test.csv")

我得到输出:

>sentiment_matrix_final = data.frame (Comment = review_phrases)

> df_scores = data.frame(Scores = review_scores)

> sentiment_matrix_final = cbind(sentiment_matrix_final, df_scores)
Error: cannot allocate vector of size 750 Kb

0 个答案:

没有答案