使用Solr Data Import Handler从mySQL导入多值字段到Solr

时间:2013-11-27 04:20:34

标签: mysql solr solr4 dataimporthandler

我们的mySQL中有以下两个表:

mysql> describe comment;
+--------------+--------------+------+-----+---------+-------+
| Field        | Type         | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+-------+
| id           | int(11)      | YES  |     | NULL    |       |
| blogpost_id  | int(11)      | YES  |     | NULL    |       |
| comment_text | varchar(256) | YES  |     | NULL    |       |
+--------------+--------------+------+-----+---------+-------+

mysql> describe comment_tags;
+------------+-------------+------+-----+---------+-------+
| Field      | Type        | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+-------+
| comment_id | int(11)     | YES  |     | NULL    |       |
| tag        | varchar(80) | YES  |     | NULL    |       |
+------------+-------------+------+-----+---------+-------+

每个评论可以有多个标签。我们可以使用数据导入处理程序将整个注释导入Solr。但是,我不确定如何将每个注释的标记导入到为每个注释文档定义schema.xml的多值字段中。

请指教。感谢

3 个答案:

答案 0 :(得分:12)

你也可以将GROUP_CONCAT与分隔符(例如“,”)一起使用,然后尝试这样的事情:

<dataConfig>
<!-- dataSource is just an example. Included just for completeness. -->
 <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/db" user="root" password="root"/>
   <document>
     <entity name="comment" pk="id" query="SELECT *, group_concat(tags) as comment_tags FROM comment" transformer="RegexTransformer">
      <field column="blogpost_id" name="blogpost_id"/>
      <field column="comment_text" name="comment_text" />
      <field column="tag" name="comment_tags" splitBy = "," />       
    </entity>
  </document>    
</dataConfig>  

它会提高性能,也会删除另一个查询的依赖关系。

答案 1 :(得分:5)

尝试这样的事情:

<dataConfig>
    <!-- dataSource is just an example. Included just for completeness. -->
    <dataSource batchSize="500" type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/my-database" user="root" password="somethinglong1283"/>
<document>
    <entity name="comment" pk="id" query="SELECT * FROM comment">
        <field column="blogpost_id" name="blogpost_id"/>
        <field column="comment_text" name="comment_text" />
        <entity name="comment_tags" pk="comment_id" query="SELECT * FROM comment_tags WHERE comment_id='${comment.id}'">
            <field column="tag" name="tag" />
        </entity>
    </entity>
</document>

答案 2 :(得分:2)

如果其他解决方案不起作用,请尝试使用此解决方案。

<dataConfig>
<!-- dataSource is just an example. Included just for completeness. -->
 <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/db" user="root" password="root"/>
   <document>
     <entity name="comment" pk="id" query="SELECT *, group_concat(tags) as tag FROM comment" transformer="RegexTransformer">
      <field column="blogpost_id" name="blogpost_id"/>
      <field column="comment_text" name="comment_text" /> 
      <field column="tag" splitBy="," sourceColName="tag"/>     
    </entity>
  </document>    
</dataConfig>

schema.xml

中添加字段
<field name="tag" type="string" indexed="true" stored="true" multiValued="true"/>

如果您想在 mysql 中使用自定义分隔符,请使用下面的。

GROUP_CONCAT(tags SEPARATOR '~,~') AS tags

如果你想在concat标签中DISTINCT那么

GROUP_CONCAT(DISTINCT tags SEPARATOR '~,~') AS tags