Question

我在这个特定的系列中有300,000个文件。每份文件都被视为一次出租车旅行。每个文档都包含TaxiStation编号和许可证编号。

我的目标是计算每辆TaxiStation的TaxiLicense旅行次数例如：
TaxiStation A许可证X有5次旅行 TaxiStation A许可Y有9次旅行。等等。

如何优化查询？这需要30分钟的完成时间才能完成！

List /*of*/ taxistationOfCollection, taxiLicenseOfTaxistation;
        //Here I get all the distinct TaxiStation numbers in the collection
        taxistationOfCollection = coll.distinct("TaxiStation");

        BasicDBObject query, tripquery;
        int tripcount;

        //Now I have to loop through each Taxi Station
        for(int i = 0; i<taxistationOfCollection.size(); i++)
        {
            query = new BasicDBObject("TaxiStation", taxistationOfCollection.get(i));
            //Here, I make a list of each distinct Taxi License in the current Taxi station
            taxiLicenseOfTaxistation = coll.distinct("TaxiLicense", query);

            //Now I make a loop to process each Taxi License within the current Taxi station
            for(int k = 0; k<taxiLicenseOfTaxistation.size();k++)
            {
                tripcount=0;
                if(taxiLicenseOfTaxistation.get(k) !=null)
                {
                    //I'm looking for each Taxi Station with this Taxi License
                    tripquery= new BasicDBObject("TaxiStation", taxistationOfCollection.get(i)).append("TaxiLicense", taxiLicenseOfTaxistation.get(k));
                    DBCursor cursor = coll.find(tripquery);

                    try {
                        while(cursor.hasNext()) {
                            //Increasing my counter everytime I find a match
                            tripcount++;
                            cursor.next();
                        } 
                    } finally {
                        //Finally printing the results
                        System.out.println("Station: " + taxistationOfCollection.get(i) + " License:" + taxiLicenseOfTaxistation.get(k)
                                + " Trips: " + tripcount);
                    }



                }
            }
        }

示例文档：

{
  "_id" : ObjectId("53df46ed9b2ed78fb7ca4f23"),
  "Version" : "2",
  "Display" : [],
  "Generated" : "2014-08-04,16:40:05",
  "GetOff" : "2014-08-04,16:40:05",
  "GetOffCellInfo" : "46001,43027,11237298",
  "Undisplay" : [],
  "TaxiStation" : "0000",
  "GetOn" : "2014-08-04,16:40:03",
  "GetOnCellInfo" : "46001,43027,11237298",
  "TaxiLicense" : "000000",
  "TUID" : "26921876-3bd5-432e-a014-df0fb26c0e6c",
  "IMSI" : "460018571356892",
  "MCU" : "CM8001MA121225V1",
  "System_ID" : "000",
  "MeterGetOffTime" : "",
  "MeterGetOnTime" : "",
  "Setup" : [],
  "MeterSID" : "",
  "MeterWaitTime" : "",
  "OS" : "4.2",
  "PackageVersion" : "201407300888",
  "PublishVersion" : "201312060943",
  "SWVersion" : "rel_touchbox_20101010",
  "MeterMile" : 0,
  "MeterCharged" : 0,
  "GetOnLongitude" : 0,
  "GetOnLatitude" : 0,
  "GetOffLongitude" : 0,
  "TripLength" : 2,
  "GetOffLatitude" : 0,
  "Clicks" : 0,
  "updateTime" : "2014-08-04 16:40:10"
}

Answer 1

Aggregation可能就是你要找的东西。通过聚合操作，您的整个代码可以在数据库上运行，并且可以在几行中执行。性能也应该更好，因为数据库处理需要完成的所有事情，可以充分利用索引和其他东西。

从你提出的这个归结为simple $group operation。在shell中，这看起来像：

db.taxistationOfCollection.aggregate([
                         {$group: 
                             { _id:
                                    {station: "$TaxiStation", 
                                    licence: "$TaxiLicense"},
                              count : {$sum : 1}
                          }
                        ])

这将为您提供表格

的文件

{_id : {station: stationid, licence: licence_number}, count: number_of_documents}

对于Java，它看起来像这样：

 DBObject taxigroup = new BasicDBObject("$group",
                               new BasicDBObject("_id", 
                                   new BasicDBObject("station","$TaxiStation")
                                   .append("Licence","$TaxiLicense"))
                               .append("count", new BasicDBObject("$sum",1)));
AggregationOutput aggout = taxistationOfCollection.aggregate(
                                                      Arrays.asList(taxigroup));

请注意，代码段未经过测试。

如何优化Mongodb的查询

1 个答案: