如何构建和使用flink-connector-kinesis?

时间:2018-12-19 09:00:31

标签: amazon-web-services maven apache-flink amazon-kinesis connector

我正在尝试将Apache Flink与AWS kinesis结合使用。 document说,我必须自己构建连接器。

因此,我建立了连接器并为项目添加了jar文件,并且还将依赖项放在了pom.xml文件中。

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-connector-kinesis_2.11</artifactId>
    <version>1.6.1</version> 
</dependency>

但是,当我尝试使用mvn clean package进行构建时,出现了这样的错误消息

[INFO] -----------------------< kkaldk:flink-kinesis >-----------------------
[INFO] Building Flink Quickstart Job 0.1
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for org.apache.flink:flink-connector-kinesis_2.11:jar:1.6-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.458 s
[INFO] Finished at: 2018-12-19T17:45:43+09:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project flink-kinesis: Could not resolve dependencies for project kkaldk:flink-kinesis:jar:0.1: Failure to find org.apache.flink:flink-connector-kinesis_2.11:jar:1.6-SNAPSHOT in https://repository.apache.org/content/repositories/snapshots/ was cached in the local repository, resolution will not be reattempted until the update interval of apache.snapshots has elapsed or updates are forced -> 
[Help 1]
[ERROR] 

您能帮我解决这个问题吗?

---------添加----------

这就是我所做的(这是错误的过程)

  1. 在此document
  2. 之后开始了项目
  3. flink master source构建连接器(1.8快照)
  4. 添加1.6.1版本的依赖项。

1 个答案:

答案 0 :(得分:4)

这就是我为解决方案所做的事情。

  1. 我从enter image description here下载了1.6.1版本。

  2. 解压缩文件(档案将在.../flink-release-1.6.1/

  3. 中解压缩
  4. mvn clean install -Pinclude-kinesis -DskipTests

  5. 构建

然后我的本地Maven存储库中有flink-connector-kinesis_2.11版本1.6.1

(对我来说,路径是这样的):~/.m2/repository/org/apache/flink/flink-connector-kinesis_2.11

然后,通过在此项目中添加依赖项,我可以在任何项目中使用此连接器。 (我不必再添加jar文件。)

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-connector-kinesis_2.11</artifactId>
    <version>1.6.1</version> 
</dependency>