带有Kafka + Confluent架构注册表客户端的Spring Cloud Stream损坏了吗?

时间:2017-05-05 16:37:23

标签: spring apache-kafka avro spring-cloud-stream confluent

如果有人因为我目前正在努力工作而感到好奇。

我创建了简单的Source和Sink应用程序来发送和接收基于Avro架构的消息。消息的架构保存在Confluent架构注册表中。这两个应用程序都配置为使用ConfluentSchemaRegistryClient类,但我认为这里可能存在一个错误。这就是我所看到的让我惊讶的事情。

如果我与Confluent注册表的REST API进行交互,我可以看到只有一个版本的架构(轻微编辑以模糊我正在处理的内容):

$ curl -i "http://schemaregistry:8081/subjects/somesubject/versions"
HTTP/1.1 200 OK
Date: Fri, 05 May 2017 16:13:37 GMT
Content-Type: application/vnd.schemaregistry.v1+json
Content-Length: 3
Server: Jetty(9.2.12.v20150709)

[1]

当Source应用程序通过Kafka发送消息时,我注意到标题中的版本看起来有点时髦:

contentType"application/octet-stream"originalContentType/"application/vnd.somesubject.v845+avro"

我并非100%清楚application/vnd.somesubject.v845+avro内容类型为何被application/octet-stream包裹但忽略了这一点,请注意它是说版本845不是版本1.

查看ConfluentSchemaRegistryClient实现,我看到它POST/subjects/(string: subject)/versions并返回架构的 id 而不是版本。然后将其放入SchemaReference version字段:https://github.com/spring-cloud/spring-cloud-stream/blob/master/spring-cloud-stream-schema/src/main/java/org/springframework/cloud/stream/schema/client/ConfluentSchemaRegistryClient.java#L81

当Sink应用程序尝试根据标头获取邮件的架构时,它会失败,因为它尝试获取版本845,它从标题中拔出:https://github.com/spring-cloud/spring-cloud-stream/blob/master/spring-cloud-stream-schema/src/main/java/org/springframework/cloud/stream/schema/client/ConfluentSchemaRegistryClient.java#L87

有人对此有何想法?提前谢谢。

**更新**

确定这是一个错误。取ConfluentSchemaRegistryClient并将register方法略微修改为POST/subjects/(string: subject)(即删除了尾随/versions),每个Confluent REST API文档返回一个有效负载其中version。像魅力一样:

public SchemaRegistrationResponse register(String subject, String format, String schema) {
    Assert.isTrue("avro".equals(format), "Only Avro is supported");
    String path = String.format("/subjects/%s", subject);
    HttpHeaders headers = new HttpHeaders();
    headers.put("Accept",
            Arrays.asList("application/vnd.schemaregistry.v1+json", "application/vnd.schemaregistry+json",
                    "application/json"));
    headers.add("Content-Type", "application/json");
    Integer version = null;
    try {
        String payload = this.mapper.writeValueAsString(Collections.singletonMap("schema", schema));
        HttpEntity<String> request = new HttpEntity<>(payload, headers);
        ResponseEntity<Map> response = this.template.exchange(this.endpoint + path, HttpMethod.POST, request,
                Map.class);
        version = (Integer) response.getBody().get("version");
    }
    catch (JsonProcessingException e) {
        e.printStackTrace();
    }
    SchemaRegistrationResponse schemaRegistrationResponse = new SchemaRegistrationResponse();
    schemaRegistrationResponse.setId(version);
    schemaRegistrationResponse.setSchemaReference(new SchemaReference(subject, version, "avro"));
    return schemaRegistrationResponse;
}

0 个答案:

没有答案