Question

我想将1000000个文档插入RavenDB。

class Program
{
        private static string serverName;
        private static string databaseName;

        private static DocumentStore documentstore;
        private static IDocumentSession _session;

        static void Main(string[] args)
        {

            Console.WriteLine("Start...");

            serverName = ConfigurationManager.AppSettings["ServerName"];
            databaseName = ConfigurationManager.AppSettings["Database"];

            documentstore = new DocumentStore { Url = serverName };
            documentstore.Initialize();

            Console.WriteLine("Initial Databse...");

            _session = documentstore.OpenSession(databaseName);

            for (int i = 0; i < 1000000; i++)
            {
                var person = new Person()    
                {
                    Fname = "Meysam" + i,
                    Lname = " Savameri" + i,
                    Bdate = DateTime.Now,
                    Salary = 6001 + i,
                    Address = "BITS provides one foreground and three background priority levels that" +
                              "you can use to prioritize transBfer jobs. Higher priority jobs preempt"+
                              "lower priority jobs. Jobs at the same priority level share transfer time,"+
                              "which prevents a large job from blocking small jobs in the transfer"+
                              "queue. Lower priority jobs do not receive transfer time until all the "+
                              "higher priority jobs are complete or in an error state. Background"+
                              "transfers are optimal because BITS uses idle network bandwidth to"+
                              "transfer the files. BITS increases or decreases the rate at which files "+
                              "are transferred based on the amount of idle network bandwidth that is"+
                              "available. If a network application begins to consume more bandwidth,"+
                              "BITS decreases its transfer rate to preserve the user's interactive"+
                              "experience. BITS supports multiple foreground jobs and one background"+
                              "transfer job at the same time.",
                    Email = "Meysam" + i + "@hotmail.com",
                };

                _session.Store(person);

                Console.ForegroundColor = ConsoleColor.Green;
                Console.WriteLine("Count:" + i);
                Console.ForegroundColor = ConsoleColor.White;
            }

            Console.WriteLine("Commit...");

            _session.SaveChanges();
            documentstore.Dispose();

            _session.Dispose();

            Console.WriteLine("Complete...");
            Console.ReadLine();
        }
    }

但是会话没有保存更改，我收到错误：

mscorlib.dll中出现未处理的“System.OutOfMemoryException”类型异常

Answer 1

document session旨在处理少量请求。相反，尝试以1024的批量插入。之后，处理会话并创建一个新会话。获得OutOfMemoryException的原因是因为文档会话缓存所有组成对象以提供unit of work，这就是您在插入批处理后应该处理会话的原因。

一种巧妙的方法是使用Batch linq extension：

foreach (var batch in Enumerable.Range(1, 1000000)
 .Select(i => new Person { /* set properties */ })
 .Batch(1024))
{
 using (var session = documentstore.OpenSession())
 {
   foreach (var person in batch)
   {
     session.Store(person);
   }
   session.SaveChanges();
 }
}

Enumerable.Range和Batch的实现都是惰性的，不会将所有对象保留在内存中。

Answer 2

RavenDB还有一个bulk API，可以做类似的事情而不需要额外的LINQ扩展：

using (var bulkInsert = store.BulkInsert())
{
    for (int i = 0; i < 1000 * 1000; i++)
    {
        bulkInsert.Store(new User
            {
                Name = "Users #" + i
            });
    }
}

注意.SaveChanges()未被调用，将在达到批量大小时调用（如果需要，可在BulkInsert()中定义），或者在bulkInsert处置时调用{{1}}

将1000000个文档插入RavenDB

2 个答案: