展开和聚合的MongoDB性能

时间：2018-10-28 16:50:13

标签： c# mongodb

我正在尝试改进查询以返回数组（“ Facets”）中每个项目的计数。 其他查询看起来不错。

编辑：添加3个示例文档：

{ 
"_id" : ObjectId("5bab1d5e2172eda710338c5c"), 
"SiteID" : "VR_1038936695_1", 
"PriceSale" : 580000.0, 
"Title" : "Apartamento a Venda em Salvador, Pituba, 4 dormitórios, 2 suítes, 4 banheiros, 2 vagas", 
"Description" : "Apartamento 44 dormitórios (sendo 2 suítes), banheiros, 2 garagens, dependência de empregada, sala integrada à varanda.andar alto, 119 mº. Condomínio com infraestrutura completa: Piscina, quadra poliesportiva, academia, salão de festas, brinquedoteca, parque infantil, salão de jogos, playground com bastante área. Localização: Próximo ao Hiper Ideal, escolas, faculdade, Mini Shopping, etc... <br> <br> OPORTUNIDADE!!! <br> <br> Agende Sua Visita!!! <br> <br> <br> - Ar Condicionado <br> - Móveis Planejados <br> - Portão Eletrônico <br> - Área de Serviço <br> - Cozinha <br> - Bares e Restaurantes <br> - Escola <br> - Farmácia <br> - Shopping Center <br> - Supermercado", 
"Link" : "https://www.vivareal.com.br/imovel/apartamento-4-quartos-pituba-bairros-salvador-com-garagem-119m2-venda-RS580000-id-1038936695/", 
"QtyRoomsMin" : 4.0, 
"QtyRoomsMax" : 4.0, 
"QtySuitesMin" : 2.0, 
"QtySuitesMax" : 2.0, 
"QtyParkingSlotMin" : 2.0, 
"QtyParkingSlotMax" : 2.0, 
"AreaMin" : 119.0, 
"AreaMax" : 119.0, 
"QtyBathroomsMin" : 4.0, 
"QtyBathroomsMax" : 4.0, 
"SiteOrigin" : NumberInt(3), 
"Type" : NumberInt(1), 
"Subtype" : NumberInt(7), 
"UpdateDate" : ISODate("2018-10-24T00:00:51.553+0000"), 
"SortOrder" : NumberInt(280), 
"IdDistrict" : NumberInt(1876), 
"DistrictName" : "Pituba", 
"IdCity" : NumberInt(988), 
"CityName" : "Salvador", 
"IdState" : NumberInt(5), 
"StateName" : "Bahia", 
"UF" : "BA", 
"FullAddress" : "Rua Ceará", 
"ZipCode" : NumberInt(41830450), 
"Latitude" : null, 
"Longitude" : null, 
"ImageUrl" : "https://resizedimgs.vivareal.com/crop/360x240/vr.images.sp/7ff41293883888840d992d9832e7c542.jpg", 
"IdTransaction" : NumberInt(1), 
"ExpireAt" : ISODate("2018-11-12T23:00:51.553+0000"), 
"Facets" : [
    "Academia", 
    "Ar Condicionado", 
    "Área de Serviço", 
    "Cozinha", 
    "Espaço Verde / Parque", 
    "Piscina", 
    "Quadra Poliesportiva", 
    "Salão de jogos", 
    "Garagem"
]
}

{ 
"_id" : ObjectId("5bbb1ea72172eda710cfb6ff"), 
"SiteID" : "VR_56466679_1", 
"PriceSale" : 610000.0, 
"Title" : "LINDO APARTAMENTO , VENHA CONHECER!!", 
"Description" : "IMPERDÍVEL. CHARME E BOM GOSTO. EXCELENTE LOCALIZAÇÃO. LINDO APARTAMENTO. PERTO DE TUDO. TODOS OS AMBIENTES COM PLANEJADOS , SALA PARA 2 AMBIENTES ,BANHEIROS COM BOX DE VIDRO , TETO REBAIXADO COM NOVA ILUMINAÇÃO E FIAÇÃO ELETRICA", 
"Link" : "https://www.vivareal.com.br/imovel/apartamento-3-quartos-vila-regente-feijo-zona-leste-sao-paulo-com-garagem-83m2-venda-RS610000-id-56466679/", 
"QtyRoomsMin" : 3.0, 
"QtyRoomsMax" : 3.0, 
"QtySuitesMin" : 1.0, 
"QtySuitesMax" : 1.0, 
"QtyParkingSlotMin" : 2.0, 
"QtyParkingSlotMax" : 2.0, 
"AreaMin" : 83.0, 
"AreaMax" : 83.0, 
"QtyBathroomsMin" : 2.0, 
"QtyBathroomsMax" : 2.0, 
"SiteOrigin" : NumberInt(3), 
"Type" : NumberInt(1), 
"Subtype" : NumberInt(7), 
"UpdateDate" : ISODate("2018-10-14T15:21:11.618+0000"), 
"SortOrder" : NumberInt(689), 
"IdDistrict" : NumberInt(42984), 
"DistrictName" : "Vila Regente Feijó", 
"IdCity" : NumberInt(9668), 
"CityName" : "São Paulo", 
"IdState" : NumberInt(26), 
"StateName" : "São Paulo", 
"UF" : "SP", 
"FullAddress" : "Rua Água Rasa - 83", 
"ZipCode" : NumberInt(3343010), 
"Latitude" : null, 
"Longitude" : null, 
"ImageUrl" : "https://resizedimgs.vivareal.com/crop/360x240/vr.images.sp/d06c1ae285d789baa4fc8661656dbe4a.jpg", 
"IdTransaction" : NumberInt(1), 
"ExpireAt" : ISODate("2018-10-29T15:21:11.618+0000"), 
"Facets" : [
    "Espaço Verde / Parque", 
    "Academia", 
    "Área de Serviço", 
    "Garagem", 
    "Salão de Festas", 
    "Piscina", 
    "Quadra Poliesportiva"
]
}

{ 
"_id" : ObjectId("5bbb51d82172eda710db154c"), 
"SiteID" : "VR_1037226651_1", 
"PriceSale" : 2990000.0, 
"Title" : "Apartamento em São Paulo", 
"Description" : "Lindo(a) Apartamento de 278,44 m² no bairro Anália Franco na cidade de São Paulo - SP.Com 3 dormitório(s), sendo 3 suite(s), 4 banheiro(s), 1 sala(s), 1 cozinha(s), 6 vagas(s) de garagem.Condições de Pagamento:Caracteristicas:Lazer:ChurrasqueiraPlay GroundPiscinaQuadra de TenisSoláriumSalão de Festa com Espaço GourmetSalão de FestasSalão de JogosSala de GinásticaSegurança:Cameras de VideosCerca ElétricaGuaritaPortão AutomáticoVigiaPortaria", 
"Link" : "https://www.vivareal.com.br/imovel/apartamento-3-quartos-vila-regente-feijo-zona-leste-sao-paulo-com-garagem-278m2-venda-RS2990000-id-1037226651/", 
"QtyRoomsMin" : 3.0, 
"QtyRoomsMax" : 3.0, 
"QtySuitesMin" : 3.0, 
"QtySuitesMax" : 3.0, 
"QtyParkingSlotMin" : 6.0, 
"QtyParkingSlotMax" : 6.0, 
"AreaMin" : 278.0, 
"AreaMax" : 278.0, 
"QtyBathroomsMin" : 4.0, 
"QtyBathroomsMax" : 4.0, 
"SiteOrigin" : NumberInt(3), 
"Type" : NumberInt(1), 
"Subtype" : NumberInt(7), 
"UpdateDate" : ISODate("2018-10-14T15:21:19.588+0000"), 
"SortOrder" : NumberInt(896), 
"IdDistrict" : NumberInt(42984), 
"DistrictName" : "Vila Regente Feijó", 
"IdCity" : NumberInt(9668), 
"CityName" : "São Paulo", 
"IdState" : NumberInt(26), 
"StateName" : "São Paulo", 
"UF" : "SP", 
"FullAddress" : "Rua José Alexandre Almeida Luiz", 
"ZipCode" : NumberInt(3337005), 
"Latitude" : null, 
"Longitude" : null, 
"ImageUrl" : "https://resizedimgs.vivareal.com/crop/360x240/vr.images.sp/2ddd2c0162262a0ab921fbac9b0ee072.jpg", 
"IdTransaction" : NumberInt(1), 
"ExpireAt" : ISODate("2018-10-29T15:21:19.588+0000"), 
"Facets" : [
    "Cozinha"
]
}

结果是这样的：

{
  Academia: 98473
  Espaço Verde / Parque: 93840
  Cozinha: 75934
}

我的收藏集中有900万个文档，而该匹配项找到了400万个文档。

此查询需要5到12秒才能返回。太糟糕了：（

有什么方法可以改善这一点吗？

在C＃中我有这个：

var pipeline = this.Collection.Aggregate(new AggregateOptions { AllowDiskUse 
= true })
.Match(filter).SortBy(p=>p.SortOrder)
.Unwind<Property,object>(x => x.Facets)
.SortByCount<object>("$Facets");
List<FacetCount> listFacets = new List<FacetCount>();
var output = pipeline.ToList();

MongoDB中的相同查询：

aggregate([
  {
    "$match": {
      "Subtype": {
        "$in": [
          7
        ]
      },
      "IdTransaction": 1,
      "IdCity": 9668
    }
  },
  {
    "$sort": {
      "SortOrder": 1
    }
  },
  {
    "$unwind": "$Facets"
  },
  {
    "$sortByCount": "$Facets"
  }
])

这就是解释给我看的东西。

{
  "stages": [
    {
      "$cursor": {
        "query": {
          "Subtype": {
            "$in": [
              7
            ]
          },
          "IdTransaction": 1,
          "IdCity": 9668
        },
        "fields": {
          "Facets": 1,
          "_id": 0
        },
        "queryPlanner": {
          "plannerVersion": 1,
          "namespace": "SonarImovel.Property",
          "indexFilterSet": false,
          "parsedQuery": {
            "$and": [
              {
                "IdCity": {
                  "$eq": 9668
                }
              },
              {
                "IdTransaction": {
                  "$eq": 1
                }
              },
              {
                "Subtype": {
                  "$eq": 7
                }
              }
            ]
          },
          "winningPlan": {
            "stage": "FETCH",
            "inputStage": {
              "stage": "IXSCAN",
              "keyPattern": {
                "Subtype": 1,
                "IdTransaction": 1,
                "IdCity": 1,
                "AreaMax": 1
              },
              "indexName": "Query by City - Sort Area",
              "isMultiKey": false,
              "multiKeyPaths": {
                "Subtype": [

                ],
                "IdTransaction": [

                ],
                "IdCity": [

                ],
                "AreaMax": [

                ]
              },
              "isUnique": false,
              "isSparse": false,
              "isPartial": false,
              "indexVersion": 2,
              "direction": "forward",
              "indexBounds": {
                "Subtype": [
                  "[7.0, 7.0]"
                ],
                "IdTransaction": [
                  "[1.0, 1.0]"
                ],
                "IdCity": [
                  "[9668.0, 9668.0]"
                ],
                "AreaMax": [
                  "[MinKey, MaxKey]"
                ]
              }
            }
          },
          "executionStats": {
            "executionSuccess": true,
            "nReturned": 1073562,
            "executionTimeMillis": 4049,
            "totalKeysExamined": 1073562,
            "totalDocsExamined": 1073562,
            "executionStages": {
              "stage": "FETCH",
              "nReturned": 1073562,
              "executionTimeMillisEstimate": 3507,
              "works": 1073563,
              "advanced": 1073562,
              "needTime": 0,
              "needYield": 0,
              "saveState": 8478,
              "restoreState": 8478,
              "isEOF": 1,
              "invalidates": 0,
              "docsExamined": 1073562,
              "alreadyHasObj": 0,
              "inputStage": {
                "stage": "IXSCAN",
                "nReturned": 1073562,
                "executionTimeMillisEstimate": 1303,
                "works": 1073563,
                "advanced": 1073562,
                "needTime": 0,
                "needYield": 0,
                "saveState": 8478,
                "restoreState": 8478,
                "isEOF": 1,
                "invalidates": 0,
                "keyPattern": {
                  "Subtype": 1,
                  "IdTransaction": 1,
                  "IdCity": 1,
                  "AreaMax": 1
                },
                "indexName": "Query by City - Sort Area",
                "isMultiKey": false,
                "multiKeyPaths": {
                  "Subtype": [

                  ],
                  "IdTransaction": [

                  ],
                  "IdCity": [

                  ],
                  "AreaMax": [

                  ]
                },
                "isUnique": false,
                "isSparse": false,
                "isPartial": false,
                "indexVersion": 2,
                "direction": "forward",
                "indexBounds": {
                  "Subtype": [
                    "[7.0, 7.0]"
                  ],
                  "IdTransaction": [
                    "[1.0, 1.0]"
                  ],
                  "IdCity": [
                    "[9668.0, 9668.0]"
                  ],
                  "AreaMax": [
                    "[MinKey, MaxKey]"
                  ]
                },
                "keysExamined": 1073562,
                "seeks": 1,
                "dupsTested": 0,
                "dupsDropped": 0,
                "seenInvalidated": 0
              }
            }
          }
        }
      },
      {
        "$unwind": {
          "path": "$Facets"
        }
      },
      {
        "$group": {
          "_id": "$Facets",
          "count": {
            "$sum": {
              "$const": 1
            }
          }
        }
      },
      {
        "$sort": {
          "sortKey": {
            "count": -1
          }
        }
      }
    ],
    "ok": 1
  }

仅此而已：

MY VM（在Azure上） 有8GB 使用SSD运行 处理器Intel®Xeon（R）CPU E5-2673 v4 @ 2.30GHz，2295 Mhz，1 Core（s），2 Logical Processor（s）

1 个答案:

答案 0 :(得分：0)

解释计划的输出表明查询目标是最佳的。但是查询也正在扫描很多需要IO的文档。涵盖查询应提高防止文档扫描的性能，这意味着将不需要访问磁盘。有关详细信息，请参见https://docs.mongodb.com/manual/core/query-optimization/#covered-query。

此外，您可能希望使用索引来排序和删除排序阶段。 https://docs.mongodb.com/manual/tutorial/sort-results-with-indexes/index.html。

相关问题
最新问题