Question

我在Node.js中遇到内存不足的问题，并且在检查堆的快照时看到很多无法被垃圾回收的大字符串。

我使用lowDB，这些字符串主要是lowDb文件的内容。

原则上的问题...

当我使用FileAsync（因此向文件的写入是异步的）并且我执行了很多（即发即忘）写入操作时...我的堆空间是否充满了等待堆栈的条目，它们都在等待文件系统完成写入？（并且节点可以为每次完成的写入清除内存）。

当我使用lowDB保存我执行的算法的日志消息时，我做了很多写操作。稍后，我想查找特定执行的日志消息。所以基本上：

{ 
  executions: [
    {
      id: 1,
      logEvents: [...]
    },
    {
      id: 2,
      logEvents: [...]
    },
    ...       
  ]
}

我对节点处理的简化图是：

我的脚本是堆栈中的下一个脚本，并且可以运行
每次写东西都在等待文件系统返回答案
这件事使我的记忆肿，每个“事”都保存了lowdb文件的全部内容（多次？！）

尝试的打字稿代码示例：

import * as lowDb from 'lowdb';
import * as FileAsync from 'lowdb/adapters/FileAsync';

/* first block just for generating random data... */
const characters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
const charactersLength = characters.length;
const alphanum = (length: number) => {
    const result = new Buffer(length);
    for (let i = 0; i < length; i++ ) {
        result.write(characters.charAt(Math.floor(Math.random() * charactersLength)));
    }
    return result.toString('utf8');
};

class TestLowDb {
    private adapter = new FileAsync('test.json');
    private db;

    /* starting the db up, loading with Async FileAdapter */
    async startDb(): Promise<void> {
        return lowDb(this.adapter).then(db => {
            this.db = db;
            return this.db.defaults({executions: [], dbCreated: new Date()}).write().then(_ => {
                console.log('finished with intialization');
            })
        });
    }

    /* fill the database with data, fails quite quickly, finally produces a json like the following:
    * { "executions": [ { "id": "<ID>", "data": [ <data>, <data>, ... ] }, <nextItem>, ... ] } */
    async fill(): Promise<void> {
        for (let i = 0; i < 100; i++) {
            const id = alphanum(3);
            this.start(id); // add the root id for this "execution"
            for (let j = 0; j < 100; j++) {
                this.fireAndForget(id, alphanum(1000));
                // await this.wait(id, alphanum(1000));
            }
        }
    }

    /* for the first item in the list add the id with the empty array */
    start(id:string): void {
        this.db.get('executions')
            .push({id, data:[]})
            .write();
    }

    /* ignores the promise and continues to work */
    fireAndForget(id:string, data:string): void {
        this.db.get('executions')
            .find({id})
            .get('data')
            .push(data)
            .write();
    }

    /* returns the promise that the caller can handle it "properly" */
    async wait(id:string, data:string): Promise<void> {
        return this.db.get('executions')
            .find({id})
            .get('data')
            .push(data)
            .write();
    }
}

const instance = new TestLowDb();
instance.startDb().then(_ => {
    instance.fill()
});
enter code here

lowDb-内存不足

0 个答案: