使用Slow Down性能解决Overembedding MongoDB问题

时间:2016-09-09 19:49:47

标签: mongodb redis data-warehouse node-redis real-time-data

目前我正在一个严重存在Overembedding问题的项目中工作,因为只有一个集合中有3个对象(数组),其实际上包含应用程序的70%商务模型(我们收到了这个)来自其他开发团队的项目,是一个完整的挑战)。另一个问题是,该应用程序使用实时跟踪使用地理位置,并继续使用此集合。

我的猜测是,我完全相信数据库服务器过载的问题以及应用程序在几个小时内的速度减慢是由于这个原因导致Overembedding。

我们认为解决方案是创建一个新的数据库模式(未知MongoDB是无模式的,但不是限制因素),尝试在具有低引用的树集合中规范化这三个对象(模仿像关系模型那样的外键),但是,你建议例如设计和制作一个带有旧(当前)数据库的Datawarehouse,仅用于读取查询,只迁移用户数据或将所有数据库迁移到新模型(可能非常复杂......或者不是?) ......

其他信息: 巴士收集统计

{
    "ns" : "pruebas.buses",
    "count" : 1343,
    "size" : 38393616,
    "avgObjSize" : 28587,
    "numExtents" : 7,
    "storageSize" : 58277888,
    "lastExtentSize" : 20643840.0,
    "paddingFactor" : 1.0,
    "paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
    "userFlags" : 1,
    "capped" : false,
    "nindexes" : 1,
    "totalIndexSize" : 65408,
    "indexSizes" : {
        "_id_" : 65408
    },
    "ok" : 1.0
}

这是一个名为Buses:

的集合中的Document示例
{
    "_id" : "BAOB-02",
    "school" : "BAOBAB",
    "licensePlate" : "UFS 118",
    "color" : "BLANCO",
    "model" : 2002,
    "username" : "baobab02",
    "students" : [ 
        {
            "firstNames" : "MATTHIAS ",
            "lastNames" : "GARCIA VELANDIA",
            "_id" : "1002",
            "classroom" : "",
            "blood" : "",
            "telephone" : null,
            "cellphone" : null,
            "guardians" : [ 
                {
                    "firstNames" : "GUSTAVO ",
                    "lastNames" : "GARCIA GARAVITO",
                    "_id" : ObjectId("553515248a854eba40c1d2fc")
                }, 
                {
                    "firstNames" : "CLAUDIA ",
                    "lastNames" : "VELANDIA ",
                    "_id" : ObjectId("553515248a854eba40c1d2fb")
                }
            ],
            "parents" : [ 
                {
                    "firstNames" : "GUSTAVO ",
                    "lastNames" : "GARCIA GARAVITO",
                    "telephone" : null,
                    "cellphone" : 3103247894.0,
                    "email" : "gggzipa@gmail.com",
                    "_id" : ObjectId("553515248a854eba40c1d2fe")
                }, 
                {
                    "firstNames" : "CLAUDIA ",
                    "lastNames" : "VELANDIA ",
                    "telephone" : null,
                    "cellphone" : 3102487056.0,
                    "email" : "ar.claudiavelandia@gmail.com",
                    "_id" : ObjectId("553515248a854eba40c1d2fd")
                }
            ],
            "addressInfo" : {
                "pm" : {
                    "address" : "KM 2 TABIO - CAJICA",
                    "apartment" : "",
                    "neighborhood" : "VIA TABIO",
                    "monday" : true,
                    "tuesday" : true,
                    "wednesday" : true,
                    "thursday" : true,
                    "friday" : true,
                    "saturday" : false,
                    "coords" : [ 
                        4.9242399390697, 
                        -74.0441983938217
                    ],
                    "stopOrder" : 1
                },
                "am" : {
                    "address" : "NA",
                    "apartment" : "",
                    "neighborhood" : "",
                    "monday" : false,
                    "tuesday" : false,
                    "wednesday" : false,
                    "thursday" : false,
                    "friday" : false,
                    "saturday" : false,
                    "coords" : []
                }
            },
            "code" : "1002"
        }, 
        {
            "firstNames" : "JUAN PABLO",
            "lastNames" : "ROMERO GUZMAN",
            "_id" : "1003",
            "classroom" : "",
            "blood" : "",
            "telephone" : null,
            "cellphone" : null,
            "guardians" : [ 
                {
                    "firstNames" : "NELSON ANDRES",
                    "lastNames" : "ROMERO ",
                    "_id" : ObjectId("5535158b8a854eba40c1d300")
                }, 
                {
                    "firstNames" : "ANA MARIA",
                    "lastNames" : "GUZMAN MORENO",
                    "_id" : ObjectId("5535158b8a854eba40c1d2ff")
                }
            ],
            "parents" : [ 
                {
                    "firstNames" : "NELSON ANDRES",
                    "lastNames" : "ROMERO ",
                    "telephone" : null,
                    "cellphone" : 3192997309.0,
                    "email" : "nelsonandresromerojimenez@hotmail.com",
                    "_id" : ObjectId("5535158b8a854eba40c1d302")
                }, 
                {
                    "firstNames" : "ANA MARIA",
                    "lastNames" : "GUZMAN MORENO",
                    "telephone" : null,
                    "cellphone" : 3143095644.0,
                    "email" : "ananita28@hotmail.com",
                    "_id" : ObjectId("5535158b8a854eba40c1d301")
                }
            ],
            "addressInfo" : {
                "pm" : {
                    "address" : "CRR 7 2 46",
                    "apartment" : "APT. 404 INT. 8",
                    "neighborhood" : "CAPELLANIA",
                    "monday" : true,
                    "tuesday" : true,
                    "wednesday" : true,
                    "thursday" : true,
                    "friday" : true,
                    "saturday" : false,
                    "coords" : [ 
                        4.91861203215498, 
                        -74.0340435504913
                    ],
                    "stopOrder" : 2
                },
                "am" : {
                    "address" : "NA",
                    "apartment" : "",
                    "neighborhood" : "",
                    "monday" : false,
                    "tuesday" : false,
                    "wednesday" : false,
                    "thursday" : false,
                    "friday" : false,
                    "saturday" : false,
                    "coords" : []
                }
            },
            "code" : "1003"
        }
    ],
    "auxiliary" : {
        "firstNames" : "LEIDY VIVIANA",
        "lastNames" : "MORANTES BARON",
        "telephone" : null,
        "cellphone" : 3203178186.0,
        "email" : "vivis_120490@hotmail.com"
    },
    "driver" : {
        "firstNames" : "VICTOR JULIO",
        "lastNames" : "MORANTES MORANTES",
        "telephone" : null,
        "cellphone" : 3118955381.0
    },
    "__v" : 13
}

此系列包含学生内部+ - 18,每个学生一般有2个父母。目前存在1300个文件。实时地理定位跟踪的数据在另一个集合中分配,但项目使用另一个服务器进行REDIS缓存(我知道缓存所有数据库不是一个好的做法,但我们计划仅将此缓存分段用于跟踪服务)< / p>

所有DB&gt;

的统计数据
{
    "db" : "pruebas",
    "collections" : 20,
    "objects" : 5785288,
    "avgObjSize" : 285.557788652873,
    "dataSize" : 1652034048.0,
    "storageSize" : 2388484096.0,
    "numExtents" : 112,
    "indexes" : 18,
    "indexSize" : 176544368.0,
    "fileSize" : 4226809856.0,
    "nsSizeMB" : 16,
    "extentFreeList" : {
        "num" : 0,
        "totalSize" : 0
    },
    "dataFileVersion" : {
        "major" : 4,
        "minor" : 22
    },
    "ok" : 1.0
}
PD /一个月前,我们可能会为MongoDB应用优化技术,比如使用负载均衡器和Mongos进行Sharding或复制......但是无论如何,我们都知道如果数据库设计错误,解决问题的最佳方法是制作一个新模型。 谢谢,如果有人花时间阅读所有这些奇怪的案例..并提前感谢如果提出建设性意见和建议

1 个答案:

答案 0 :(得分:1)

在上面的描述中,没有提到根本原因并且看起来基于假设,brodriguezs正在进行架构更改

在修改模式之前很少有提示。

您可以在此处看到一些重要提示 - https://docs.mongodb.com/manual/administration/analyzing-mongodb-performance/