Question

我想使用Java查询mongodbs oplog，如果可能的话，使用spring mongodb集成。我的问题是从java创建以下查询：

db['oplog.rs'].find({ "ts": { $gt: Timestamp(1489568405,34) }, $and: [ { "ns": "myns" } ] })

我尝试了一些像BsonTimestamp或BSONTimestamp这样的东西导致错误的查询。使用

BasicQuery({ "ts": { $gt: Timestamp(1489568405,34) }, $and: [ { "ns": "myns" } ] })

导致java mongodb驱动程序的JSON解析器出错。

任何提示？

ThxJürgen

典型记录如下：

{ 
    "ts" : Timestamp(1489567144, 2), 
    "t" : NumberLong(2), 
    "h" : NumberLong(7303473893196954969), 
    "v" : NumberInt(2), 
    "op" : "i", 
    "ns" : "asda.jam", 
    "o" : {
        "_id" : NumberInt(2), 
        "time" : ISODate("2017-03-15T08:39:00.000+0000"), 
        "roadDesc" : {
            "roadId" : NumberInt(28102917), 
            "roadName" : "A480 W"
        }, 
        "posUpFront" : NumberInt(1003), 
        "posDownFront" : NumberInt(1003), 
        "_class" : "de.heuboe.acaJNI.test.Jam"
    }
}

Answer 1

Mongo有一个扩展的JSON语法，用于构造像NumberLong，Timestamp等，它们可以在Mongo shell上运行。为了使它在Java代码中工作，它们具有严格的JSON模式，其中使用JSON（https://docs.mongodb.com/manual/reference/mongodb-extended-json/#bson-data-types-and-associated-representations）表示这些运算符。要使用Java执行此操作，您可以创建自定义转换器并将其注册到MappingMongoConverter中（请参阅下面的代码段）。转换器应将数据类型（例如BSONTimestamp）转换为适当的严格JSON文档格式。

@WritingConverter
public class BsonTimestampToDocumentConverter implements Converter<BSONTimestamp, Document> {

  private static final Logger LOGGER = LoggerFactory.getLogger(BsonTimestampToDocumentConverter.class);

  public BsonTimestampToDocumentConverter() {
    //
  }

  @Override
  public Document convert(BSONTimestamp source) {
    LOGGER.trace(">>>> Converting BSONTimestamp to Document");
    Document value = new Document();
    value.put("t", source.getTime());
    value.put("i", source.getInc());
    return new Document("$timestamp", value);
  }
}

像这样

在MappingMongoConverter中注册它

 public MappingMongoConverter syncLocalMappingMongoConverter() throws Exception {
    MongoMappingContext mappingContext = new MongoMappingContext();
    DbRefResolver dbRefResolver = new DefaultDbRefResolver(syncLocalDbFactory());
    MappingMongoConverter converter = new MappingMongoConverter(dbRefResolver, mappingContext);
    converter.setCustomConversions(customConversions());

    return converter;
 }


  private CustomConversions customConversions()  {
   List<Converter<?, ?>> converterList = new ArrayList<>();
   converterList.add(new BsonTimestampToDocumentConverter());
   // add the other converters here
   return new CustomConversions(CustomConversions.StoreConversions.NONE, converterList);
}

这是一个示例，我用它来查询oplog存储库以在一定时间后返回记录（存储库中的Sync用于区分它与我正在处理的反应性异步内容.Async存储库看起来完全相同除外它应该扩展ReactiveMongoRepository）。 OplogRecord类是我创建的Java bean，用于匹配MongoDb oplog记录的结构。

public interface SyncOplogRepository extends MongoRepository<OplogRecord, Long> {

  @Query(value = "{ \"op\": { $nin: ['n', 'c'] } }") List<OplogRecord> findRecordsNotEqualToNOrC();

  @Query(value = "{'ts' : {$gte : ?0}, \"op\": { $nin: ['n', 'c'] } }")
  List<OplogRecord> findRecordsNotEqualToNOrCAfterTime(BSONTimestamp timestamp);

  @Query(value = "{'ts' : {$lt : ?0}, \"op\": { $nin: ['n', 'c'] } }") 
  List<OplogRecord> findRecordsNotEqualToNOrCBeforeTime(BSONTimestamp timestamp);

}

OplogRecord类

import com.mongodb.DBObject;
import org.bson.BsonTimestamp;
import org.bson.types.BSONTimestamp;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;

import java.util.Map;


@Document(collection = "oplog.rs")
public class OplogRecord {

  @Id
  private Long id;

  /**
   * Timestamp
   */
  private BsonTimestamp ts;

  /**
   * Unique id for this entry
   */
  private Long h;

  /**
   * DB and collection name of change.
   */
  private String ns;

  /**
   * The actual document that was modified/inserted/deleted
   */
  private Map<String, Object> o;

  /**
   * The operation that was performed
   */
  private String op;

  /**
   * ??
   */
  private Long t;

  /**
   * ??
   */
  private Integer v;

  public BsonTimestamp getTs() {
    return ts;
  }

  public void setTs(BsonTimestamp ts) {
    this.ts = ts;
  }

  public Long getH() {
    return h;
  }

  public void setH(Long h) {
    this.h = h;
  }

  public String getNs() {
    return ns;
  }

  public void setNs(String ns) {
    this.ns = ns;
  }

  public Map<String, Object> getO() {
    return o;
  }

  public void setO(Map<String, Object> o) {
    this.o = o;
  }

  public String getOp() {
    return op;
  }

  public void setOp(String op) {
    this.op = op;
  }

  public Long getT() {
    return t;
  }

  public void setT(Long t) {
    this.t = t;
  }

  public Integer getV() {
    return v;
  }

  public void setV(Integer v) {
    this.v = v;
  }
}

〜
〜

Answer 2

您可以使用org.bson.BsonTimestamp进行过滤。

BsonTimestamp lastReadTimestamp = new BsonTimestamp(1489568405, 34);
Bson filter = new Document("$gt", lastReadTimestamp);

然后你可以使用find，就像这样，

oplogColl.find(new Document("ts", filter));

或者您可以创建一个可存活的游标并遍历这样的文档，

MongoCursor oplogCursor =
                    oplogColl
                            .find(new Document("ts", filter))
                            .cursorType(CursorType.TailableAwait)
                            .noCursorTimeout(true)
                            .batchSize(1000)
                            .iterator();

使用spring mongo查询oplog时间戳

2 个答案: