Question

我想使用Fiware Cygnus将历史数据保存到MongoDB中。但是，每当Cygnus收到通知时，它想要创建一个新集合，而不是在集合已经存在的情况下添加文档。我从Cygnus收到此错误消息：

time=2017-01-14T20:58:11.785Z | lvl=WARN | corr=28eca6a8-da9c-11e6-841b-0242ac120009 | trans=9fa2d345-9aa1-4ff6-84df-75de5829a449 | srv=itg | subsrv=/building1 | comp=cygnus-ngsi | op=processNewBatches | msg=com.telefonica.iot.cygnus.sinks.NGSISink[590] : Command failed with error 48: 'a collection 'sth_itg.sth_/building1_TemperatureRoom1_room' already exists' on server iot-mongo:27017. The full response is { "ok" : 0.0, "errmsg" : "a collection 'sth_itg.sth_/building1_TemperatureRoom1_room' already exists", "code" : 48, "codeName" : "NamespaceExists" }

这是我的agent.conf文件：

cygnus-ngsi.sources = http-source
cygnus-ngsi.sinks = mongo-sink
cygnus-ngsi.channels = mongo-channel

cygnus-ngsi.sources.http-source.type = org.apache.flume.source.http.HTTPSource
cygnus-ngsi.sources.http-source.channels = mongo-channel
cygnus-ngsi.sources.http-source.port = 5050
cygnus-ngsi.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.NGSIRestHandler
cygnus-ngsi.sources.http-source.handler.notification_target = /notify
cygnus-ngsi.sources.http-source.handler.default_service = default
cygnus-ngsi.sources.http-source.handler.default_service_path = /
cygnus-ngsi.sources.http-source.interceptors = ts gi
cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp
cygnus-ngsi.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder
cygnus-ngsi.sources.http-source.interceptors.gi.grouping_rules_conf_file = /opt/apache-flume/conf/grouping_rules.conf


cygnus-ngsi.sinks.mongo-sink.type = com.telefonica.iot.cygnus.sinks.NGSIMongoSink
cygnus-ngsi.sinks.mongo-sink.channel = mongo-channel
#cygnus-ngsi.sinks.mongo-sink.enable_encoding = false
#cygnus-ngsi.sinks.mongo-sink.enable_grouping = false
#cygnus-ngsi.sinks.mongo-sink.enable_name_mappings = false
#cygnus-ngsi.sinks.mongo-sink.enable_lowercase = false
#cygnus-ngsi.sinks.mongo-sink.data_model = dm-by-entity
#cygnus-ngsi.sinks.mongo-sink.attr_persistence = row
cygnus-ngsi.sinks.mongo-sink.mongo_hosts = iot-mongo:27017
cygnus-ngsi.sinks.mongo-sink.mongo_username =
cygnus-ngsi.sinks.mongo-sink.mongo_password =
#cygnus-ngsi.sinks.mongo-sink.db_prefix = sth_
#cygnus-ngsi.sinks.mongo-sink.collection_prefix = sth_
#cygnus-ngsi.sinks.mongo-sink.batch_size = 1
#cygnus-ngsi.sinks.mongo-sink.batch_timeout = 30
#cygnus-ngsi.sinks.mongo-sink.batch_ttl = 10
#cygnus-ngsi.sinks.mongo-sink.data_expiration = 0
#cygnus-ngsi.sinks.mongo-sink.collections_size = 0
#cygnus-ngsi.sinks.mongo-sink.max_documents = 0
#cygnus-ngsi.sinks.mongo-sink.ignore_white_spaces = true

cygnus-ngsi.channels.mongo-channel.type = com.telefonica.iot.cygnus.channels.CygnusMemoryChannel
cygnus-ngsi.channels.mongo-channel.capacity = 1000
cygnus-ngsi.channels.mongo-channel.transactionCapacity = 100

我做错了吗？有人可以帮忙吗？

提前谢谢。

Answer 1

在插入数据之前，Cygnus总是尝试在收到通知时创建集合。为什么？因为只有当集合不存在时才要求MongoDB进行集合存在以及要求MongoDB创建集合是通过驱动程序进行的两个操作;同时要求MongoDB始终创建集合只是一个操作。因此，如果集合存在，那么如果您看到DEBUG日志跟踪告诉您集合存在，则应该没问题。（*）

https://github.com/Vaikesh/CustomKeyboard

try {
   backend.createDatabase(dbName);
   backend.createCollection(dbName, collectionName, collectionsSize, maxDocuments, dataExpiration);
   backend.insertContextDataRaw(dbName, collectionName, aggregation);
} catch (Exception e) {
   throw new CygnusPersistenceError("-, " + e.getMessage());
} // try catch

问题是您遇到了WARN日志跟踪，这很糟糕。您使用的是哪个版本的MongoDB？回顾代码时，在创建集合时解释MongoDB响应可能会出现问题，这不是基于错误代码而是基于消息字符串：如果MongoDB返回的消息与预期不同，则会引发错误。这肯定是必须改进的。

https://github.com/telefonicaid/fiware-cygnus/blob/master/cygnus-ngsi/src/main/java/com/telefonica/iot/cygnus/sinks/NGSIMongoSink.java#L364

// create the collection
try {
   db.createCollection(collectionName);
} catch (Exception e) {
   if (e.getMessage().contains("collection already exists")) {
      LOGGER.debug("Collection already exists, nothing to create");
   } else {
      throw e;
   } // if else
} // try catch

（*）这可以通过使用已创建的数据库和集合的“缓存”来改进。这样的缓存可以通过MongoDB驱动程序节省大量操作。这个想法已经为CKAN和MySQL接收器实现了。

Fiware Cygnus总是希望在收到通知时创建一个新的MongoDB集合

1 个答案: