杜克重复数据删除引擎:无法找到确切的记录

时间:2016-10-27 08:33:59

标签: java duplicates record-linkage duke

我正在尝试为Duke创建配置和处理器,以便在记录列表中找到完全匹配。我创建了一个基于ExactMatchComparator的处理器,但该函数不返回完全匹配。 这里是处理器,配置和监听器的设置:

public void setup() {
  //setup
  List<Property> exactMatchProperties = new ArrayList<Property>();

  exactListener = new TestUtils.TestListener();

  ExactComparator exactComparator = new ExactComparator();

  //create properties (columns), with name, comparator, and high-low thresholds. ID property has no comparator or propabilities.

  exactMatchProperties.add(new PropertyImpl("ID"));
  exactMatchProperties.add(new PropertyImpl("NAME", exactComparator, 0.0, 1.0));
  exactMatchProperties.add(new PropertyImpl("EMAIL", exactComparator, 0.0, 1.0));

  //create new configuration implementation
  exactMatchConfig = new ConfigurationImpl();

  //add properties to config
  exactMatchConfig.setProperties(exactMatchProperties);
  exactMatchConfig.setThreshold(1.0);
  exactMatchConfig.setMaybeThreshold(0.0);

  //initialize the processor and add match listener
  exactMatchProcessor = new Processor(exactMatchConfig, true);
  exactMatchProcessor.addMatchListener(exactListener);
  }

这是我的测试功能:

 public void testExactMatch() {
  Collection<Record> records = new ArrayList<Record>();
  Record rec1 = TestUtils.makeRecord(new String[] { "ID", "1", "NAME", "Jon", "EMAIL", "jon@doe.com" });
  Record rec2 = TestUtils.makeRecord(new String[] { "ID", "1", "NAME", "Jon", "EMAIL", "jon@doe.com" });

  records.add(rec1);
  records.add(rec2);

  exactMatchProcessor.deduplicate(records);
  System.out.println(exactListener.getMatches().size());
}

我正在使用API​​,并且我已经阅读了有关SO here的问题,但问题是指XML,而我正在使用Java进行测试。< / p>

getMatches不应为空吗?如何获得找到的重复列表,或相反的(唯一记录列表,没有重复)? 感谢

0 个答案:

没有答案