Question

有人对File Sink Connector输出格式有疑问吗？

我想知道为什么我会收到这种格式

{op=u, before={withTax=1, unit=pc(s), deleted=0, updated_at=null, price=420000.0, name=Test 123, created_at=null, id=54, useTaxIncludedFormula=0}, after={withTax=1, unit=pc(s), deleted=0, updated_at=null, price=333.0, name=Test 123, created_at=null, id=54, useTaxIncludedFormula=0}, source={ts_sec=1561038448, query=null, thread=29638, server_id=100596, version=0.9.5.Final, file=mysql-bin.000012, connector=mysql, pos=1044011265, name=crm-kafka-connector, gtid=null, row=0, snapshot=false, db=gxapp_customer_db, table=registrationItemsTmp}, ts_ms=1561063703793}```

但是当我使用kafka-console-consumer.sh查看它时，我得到了有效的JSON

{"before":{"id":54,"name":"Test 123","price":420000.0,"unit":"pc(s)","withTax":1,"created_at":null,"updated_at":null,"useTaxIncludedFormula":0,"deleted":0},"after":{"id":54,"name":"Test 123","price":333.0,"unit":"pc(s)","withTax":1,"created_at":null,"updated_at":null,"useTaxIncludedFormula":0,"deleted":0},"source":{"version":"0.9.5.Final","connector":"mysql","name":"crm-kafka-connector","server_id":100596,"ts_sec":1561038448,"gtid":null,"file":"mysql-bin.000012","pos":1044011265,"row":0,"snapshot":false,"thread":29638,"db":"gxapp_customer_db","table":"registrationItemsTmp","query":null},"op":"u","ts_ms":1561063703793}```

我已经设置了一切以使用JsonConverter

Answer 1

FileStreamSinkConnector将根据接收到的值写入数据。 JSONConverter只是告诉Kafka Connect如何反序列化它收到的消息。

如果要转储到文件的主题上具有JSON，则在接收器连接器中使用

"value.converter":"org.apache.kafka.connect.storage.StringConverter"

此链接可能会提供更多有用的背景信息：https://rmoff.net/2019/05/08/when-a-kafka-connect-converter-is-not-a-converter/

Kafka文件接收器连接器-输出JSON

1 个答案: