Question

我们正在扩展现有节点+ mongo app。我们需要添加可能是大型文档的内容，但我们目前还不知道它们有多大。

MongoDB默认限制为最大16mb，我知道我们可以增加这个但不愿意。

有没有人见过自动doc。拆分模块？如果大小超过一定大小，会自动将文档拆分为部分内容吗？

Answer 1

If you have large CSV data to be stored in MongoDB, then there are two approaches which will both work well in different ways:

1: Save in MongoDB format

This means that you have your application read the csv, and write it to a MongoDB collection one row at a time. So each row is saved as a separate document, perhaps something like this:

{
  "filename" : "restaurants.csv",
  "version" : "2",
  "uploadDate" : ISODate("2017-06-15"),
  "name" : "Ace Cafe",
  "cuisine" : "British",
  etc
},
{
  "filename" : "restaurants.csv",
  "version" : "2",
  "uploadDate" : ISODate("2017-06-15"),
  "name" : "Bengal Tiger",
  "cuisine" : "Bangladeshi",
  etc
}

This will take work on your application's part, to render the data into this format and deciding how and where to save the metadata
You can index and query on the data, field by field and row by row
You have no worries about any single document getting too large

2: Save in CSV format using GridFS

This means that your file is uploaded as an un-analysed blob, and automatically divided into 16MB chunks in order to save it in MongoDB documents.

This is easy to do, and does not disturb your original CSV structure
However the data is opaque to MongoDB: you cannot scan it or read it row by row
to work with the data, your application will have to download the entire file from MongoDB and work on it in memory

Hopefully one of these approaches will suit your needs.

MongoDB / mongoose拆分大型文档

1 个答案:

1: Save in MongoDB format

2: Save in CSV format using GridFS