我在使用c#查询MongoDb时遇到了一个问题。问题是我没有返回正确的结果或正确的结果数。我不知道确切的结果数,但它应该小于100;相反,我收到大约350k-500k的结果(其中许多是空的)。另一个问题是该程序需要10分钟才能完成处理。
您可以在以下内容中看到有问题的代码部分:
public List<BsonDocument> find_All_Documents_With_pIDs()
{
List<string> databases = new List<string>();
List<BsonDocument> pDocs = new List<BsonDocument>();
databases.AddRange(mongo_Server.GetDatabaseNames());
//iterate through each db in mongo
foreach (string dbName in databases)
{
List<string> collections = new List<string>();
var database = mongo_Server.GetDatabase(dbName);
collections.AddRange(database.GetCollectionNames());
//iterate through each collection
foreach (string colName in collections)
{
var collection = database.GetCollection(colName);
//Iterate through each document
foreach (var document in collection.FindAllAs<BsonDocument>())
{
//Get all documents that have a pID in either the main document or its sub document
IMongoQuery query = Query.Exists(document.GetElement("_id").ToString().Remove(0,4) + ".pID");
IMongoQuery subQuery = Query.Exists(document.GetElement("_id").ToString() + ".SubDocument.pID");
pDocs.AddRange(collection.Find(query));
pDocs.AddRange(collection.Find(subQuery));
}
}
}
//Theres a collection used earlier in the program to backup the documents before processing. Not removing the documents from the list found in this location will result in duplicates.
return remove_Backup_Documents_From_List(pIDs);
}
任何帮助表示赞赏!
编辑:
以下是收到的数据的截屏。并非所有数据都如下所示为空,但非常大的数据是:
答案 0 :(得分:1)
您的脚本首先从数据库中提取所有文档
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<div class="proj">
hello
</div>
然后为每个人组装一个查询。这可能是查询速度太慢的原因。
作为替代方案,您可以执行以下操作:
collection.FindAllAs<BsonDocument>()
这样您组合的查询只返回设置了 foreach (string colName in collections)
{
var collection = database.GetCollection(colName);
//Query for all documents that have pID
IMongoQuery query = Query.And([Query.Exists("pID"), // The field exists
Query.NE("pID", BsonNull.Value), //It is not "null"
Query.NE("pID", BsonString.Null)]); //It is not empty i.e. = ""
//Query for all documents that have Subdocument.pID
IMongoQuery subQuery = Query.And([Query.Exists("SubDocument.pID"), // The field exists
Query.NE("SubDocument.pID", BsonNull.Value), //It is not "null"
Query.NE("SubDocument.pID", BsonString.Null)]); //It is not empty i.e. = ""
IMongoQuery totalQuery = Query.Or([query, subQuery]);
List<BsonDocument> results = collection.Find(totalQuery);
if (results.Count > 0) {
pDocs.AddRange(results); //Only add to pDocs if query returned at least one result
}
}
或pID
字段的文档。