我正在尝试将大量数据插入MongoDb并从集合中检索随机键值。插入大约2GB的数据需要大约29分钟,并且需要几个小时来检索键值对。与其他NoSQL数据存储相比,这非常慢。可能是瓶颈。
插入
while ((strLine = br.readLine()) != null) {
documentDetail.clear();
//String strLines = strLine.replaceAll("[-+.^:,]","");
tokens = strLine.split("\t");
documentDetail.put("key", tokens[0]);
documentDetail.put("value", tokens[1]);
start = System.currentTimeMillis();
collection.insert(documentDetail);
elapsed = System.currentTimeMillis()-start;
totalElapsed += elapsed;
}
BasicDBObject index = new BasicDBObject();
index.put("key", 1);
index.put("value", 1);
collection.ensureIndex(index);
选择
BasicDBObject find = new BasicDBObject();
find.put("value", 1);
while ((strLin = br.readLine()) != null) {
BasicDBObject query = new BasicDBObject();
query.put("key", strLin);
System.out.println(strLin);
start = System.currentTimeMillis();
DBCursor cursorDoc = collection.find(query, find);
while (cursorDoc.hasNext()) {
System.out.println(cursorDoc.next());
}
elapsed = System.currentTimeMillis()-start;
totalElapsed += elapsed;
System.out.println("Reading Ends...");
//field.clear();
}
答案 0 :(得分:1)
您的索引是否适合RAM?如果没有,MongoDB将非常不满(=慢)。
> db.collection.totalIndexSize()
要使其成为仅索引扫描/覆盖索引,请使用以下命令:
BasicDBObject query = new BasicDBObject("key",true).append("_id",false);