Spring Boot Kafka Streams - 绑定问题

时间:2018-06-05 06:55:18

标签: spring-boot spring-kafka

我正在尝试使用此示例构建基于Kafka Streams的简单流应用程序。

Word Count

然而,当我启动应用程序时,我收到以下错误:有人可以指出我在这里错过了什么吗?这是代码,配置&错误

@SpringBootApplication
@Slf4j
@EnableScheduling
@EnableBinding(PersonBinding.class)
public class DemoApplication {

  public static void main(String[] args) {
    SpringApplication.run(DemoApplication.class, args);
  }

  @Component
  public static class PersonSource {

    private final MessageChannel personOut;

    @Autowired
    PersonSource(PersonBinding personBinding) {

      this.personOut = personBinding.personOut();
    }

    @Scheduled(fixedDelay = 5000L)
    public void run() {

      Message<Person> message = MessageBuilder
          .withPayload(new Person("John", "Doe", Instant.now()))
          .build();

      try {

        personOut.send(message);

        log.info("Published message: {}", message);
      } catch (Exception e) {

        e.printStackTrace();
        throw e;
      }
    }
  }

  @Component
  public static class PersonProcessor {

    @StreamListener
    public void process(@Input(PersonBinding.PERSON_IN) KStream<String, Person> events) {

      events.foreach(((key, value) -> System.out.println("Key: " + key + "; Value: " + value)));
    }
  }
}

@Data
@AllArgsConstructor
class Person {

  String firstName;

  String lastName;

  Instant createdOn;
}

interface PersonBinding {

  String PERSON_IN = "pin";

  String PERSON_OUT = "pout";

  @Output(PERSON_OUT)
  MessageChannel personOut();

  @Input(PERSON_IN)
  KStream<String, Person> personIn();
}

依赖管理(Spring Boot 1.5.13.RELEASE)

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-stream-kafka</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-stream-binder-kstream</artifactId>
</dependency>
<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-streams</artifactId>
    <version>1.1.0</version>
</dependency>

配置

# Default Configuration
spring.cloud.stream.kstream.binder.configuration.default.key.serde=org.apache.kafka.common.serialization.Serdes$StringSerde
spring.cloud.stream.kstream.binder.configuration.default.value.serde=org.apache.kafka.common.serialization.Serdes$StringSerde
# Out Bindings Configuration
spring.cloud.stream.bindings.pout.destination=pout
spring.cloud.stream.bindings.pout.producer.header-mode=raw
# In Bindings Configuration
spring.cloud.stream.bindings.pin.destination=pout
spring.cloud.stream.bindings.pin.consumer.header-mode=raw

错误

Field configurationProperties in org.springframework.cloud.stream.binder.kafka.config.KafkaBinderConfiguration required a single bean, but 2 were found:
  - spring.cloud.stream.kafka.binder-org.springframework.cloud.stream.binder.kafka.properties.KafkaBinderConfigurationProperties: a programmatically registered singleton - binderConfigurationProperties: defined by method 'binderConfigurationProperties' in class path resource [org/springframework/cloud/stream/binder/kstream/config/KStreamBinderSupportAutoConfiguration.class]


Action:

Consider marking one of the beans as @Primary, updating the consumer to accept multiple beans, or using @Qualifier to identify the bean that should be consumed

**编辑1 **

将代码上传到Github https://github.com/tapitoe/demo-spring-cloud-streams/tree/master/src

3 个答案:

答案 0 :(得分:3)

我有一个类似的问题,我添加后得到解决:

jacobian_cost(np.array([x,x,x]))
C:\ProgramData\Anaconda3\lib\site-packages\autograd\tracer.py:48: RuntimeWarning: invalid value encountered in log
  return f_raw(*args, **kwargs)
Traceback (most recent call last):

  File "<ipython-input-94-f5cd2527501d>", line 1, in <module>
    jacobian_cost(np.array([x,x,x]))

  File "C:\ProgramData\Anaconda3\lib\site-packages\autograd\wrap_util.py", line 20, in nary_f
    return unary_operator(unary_f, x, *nary_op_args, **nary_op_kwargs)

  File "C:\ProgramData\Anaconda3\lib\site-packages\autograd\differential_operators.py", line 57, in jacobian
    vjp, ans = _make_vjp(fun, x)

  File "C:\ProgramData\Anaconda3\lib\site-packages\autograd\core.py", line 10, in make_vjp
    end_value, end_node =  trace(start_node, fun, x)

  File "C:\ProgramData\Anaconda3\lib\site-packages\autograd\tracer.py", line 10, in trace
    end_box = fun(start_box)

  File "C:\ProgramData\Anaconda3\lib\site-packages\autograd\wrap_util.py", line 15, in unary_f
    return fun(*subargs, **kwargs)

  File "<ipython-input-90-1d8956a98276>", line 3, in f
    f[0] = np.sin(x[0])+x[1]**2 +np.log(x[2])-7

TypeError: 'function' object does not support item assignment

答案 1 :(得分:1)

您需要将kstream活页夹添加到pom中;启动器只添加消息通道绑定器。

修改

我只是将类似的代码复制到应用程序中,没有任何问题。

@SpringBootApplication
@EnableBinding(So50693858Application.PersonBinding.class)
public class So50693858Application {

    public static void main(String[] args) {
        SpringApplication.run(So50693858Application.class, args);
    }

    @StreamListener
    public void process(@Input(PersonBinding.PERSON_IN) KStream<String, String> events) {

        events.foreach(((key, value) -> System.out.println("Key: " + key + "; Value: " + value)));
    }

    interface PersonBinding {

        String PERSON_IN = "pin";

        String PERSON_OUT = "pout";

        @Output(PERSON_OUT)
        MessageChannel personOut();

        @Input(PERSON_IN)
        KStream<String, String> personIn();
    }

}

并从控制台生产者发送消息到pout

Key: null; Value: foo

但是,不清楚为什么你有输入和输出绑定到同一目的地(不是那会导致你看到的问题)。

修改

这也适用于(使用您的属性):

@SpringBootApplication
@EnableBinding(So50693858Application.PersonBinding.class)
public class So50693858Application {

    public static void main(String[] args) {
        SpringApplication.run(So50693858Application.class, args);
    }

    @Bean
    public ApplicationRunner runner(MessageChannel pout) {
        return args -> {
            pout.send(new GenericMessage<>("foo".getBytes(),
                    Collections.singletonMap(KafkaHeaders.MESSAGE_KEY, "bar".getBytes())));
            pout.send(new GenericMessage<>("baz".getBytes(),
                    Collections.singletonMap(KafkaHeaders.MESSAGE_KEY, "qux".getBytes())));
        };
    }

    @StreamListener
    public void process(@Input(PersonBinding.PERSON_IN) KStream<String, String> events) {

        events.foreach(((key, value) -> System.out.println("Key: " + key + "; Value: " + value)));
    }

    interface PersonBinding {

        String PERSON_IN = "pin";

        String PERSON_OUT = "pout";

        @Output(PERSON_OUT)
        MessageChannel personOut();

        @Input(PERSON_IN)
        KStream<String, String> personIn();
    }

}

Key: bar; Value: foo
Key: qux; Value: baz

<强> EDIT3

Pom for 2.0.x版本:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>so50693858</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>so50693858</name>
    <description>Demo project for Spring Boot</description>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.0.2.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
        <spring-cloud.version>Finchley.RC2</spring-cloud.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-binder-kafka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-test-support</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-streams</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-binder-kafka-streams</artifactId>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

    <repositories>
        <repository>
            <id>spring-milestones</id>
            <name>Spring Milestones</name>
            <url>https://repo.spring.io/milestone</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>


</project>

配置:

# Default Configuration
spring.cloud.stream.kafka.streams.binder.configuration.default.key.serde=org.apache.kafka.common.serialization.Serdes$StringSerde
spring.cloud.stream.kafka.streams.binder.configuration.default.value.serde=org.apache.kafka.common.serialization.Serdes$StringSerde
# Out Bindings Configuration
spring.cloud.stream.bindings.pout.destination=pout
spring.cloud.stream.bindings.pout.producer.header-mode=raw
# In Bindings Configuration
spring.cloud.stream.bindings.pin.destination=pout
spring.cloud.stream.bindings.pin.consumer.header-mode=raw

答案 2 :(得分:0)

尝试将EnableBinding注释移至DemoApplication课程。我认为它应该放在@Configuration课程上,而不是放在任意@Component上。