Fiware天鹅座构建失败

时间:2017-04-17 22:54:57

标签: fiware fiware-cygnus

我想使用Fiware cygnus将来自上下文代理的数据保存到CKAN实例中。我已经创建了一个orion上下文代理vm,它是orion-psb-image-R5.4 1.7.0版本和Fiware云上的ckan VM。我开始安装天鹅座天鹅座知道天鹅座是基于天鹅座和天鹅座的。所以,我开始下载fiware-cygnus https://github.com/telefonicaid/fiware-cygnus 然后我开始使用maven 3.2.5成功构建cygnus-common。但是当我开始构建cygnus-ngsi时它失败了,尽管我已经尝试了很多方法来解决这个问题,但是没有人工作。 这是我在细节上所做的事情。

首先按照http://fiware-cygnus.readthedocs.io/en/1.3.0/cygnus-common/installation_and_administration_guide/install_from_sources/index.html

所述安装Apache Flume
$ wget http://www.eu.apache.org/dist/flume/1.4.0/apache-flume-1.4.0-bin.tar.gz
 tar xvzf apache-flume-1.4.0-bin.tar.gz
$ mv apache-flume-1.4.0-bin APACHE_FLUME_HOME
$ mv APACHE_FLUME_HOME/lib/httpclient-4.2.1.jar APACHE_FLUME_HOME/lib/httpclient-4.2.1.jar.old
$ mv APACHE_FLUME_HOME/lib/httpcore-4.2.1.jar APACHE_FLUME_HOME/lib/httpcore-4.2.1.jar.old
$ mv APACHE_FLUME_HOME/lib/libthrift-0.7.0.jar APACHE_FLUME_HOME/lib/libthrift-0.7.0.jar.old
$ mkdir -p APACHE_FLUME_HOME/plugins.d/cygnus/
$ mkdir APACHE_FLUME_HOME/plugins.d/cygnus/lib
$ mkdir APACHE_FLUME_HOME/plugins.d/cygnus/libext

其次,下载fiware-cygnus并建立cygnus-common:

git clone https://github.com/telefonicaid/fiware-cygnus.git

cd fiware-cygnus/cygnus-common

mvn clean compile exec:exec assembly:single

cp target/cygnus-common-1.7.0-jar-with-dependencies.jar /usr/cygnus/plugins.d/cygnus/libext 

mvn install:install-file -Dfile=/usr/cygnus/plugins.d/cygnus/libext/cygnus-common-1.7.0_SNAPSHOT-jar-with-dependencies.jar -DgroupId=com.telefonica.iot -DartifactId=cygnus-common -Dversion=1.7.0 -Dpackaging=jar -DgeneratePom=false

cp target/classes/cygnus-flume-ng /usr/cygnus/bin/cygnus-flume-ng

然后我开始按照以下方式建造天鹅座 -

cd cygnus-ngsi

mvn clean compile exec:exec assembly:single

我在构建它时获得了这些结果:

[INFO] Building cygnus-ngsi 1.7.0_SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for com.telefonica.iot:cygnus-common:jar:1.7.0_SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.243 s
[INFO] Finished at: 2017-04-17T22:04:25+00:00
[INFO] Final Memory: 7M/28M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project cygnus-ngsi: Could not resolve dependencies for project com.telefonica.iot:cygnus-ngsi:jar:1.7.0_SNAPSHOT: Could not find artifact com.telefonica.iot:cygnus-common:jar:1.7.0_SNAPSHOT -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

我将生成的jar文件复制到/usr/cygnus/plugins.d/cygnus/libext,如上面相同的链接中所述,运行:

cp cygnus-common/target/cygnus-common-1.7.0_SNAPSHOT-jar-with-dependencies.jar    /usr/cygnus/plugins.d/cygnus/libext 

我还将jar文件复制到maven存储库路径,以确保通过运行命令正确设置所有内容

cp cygnus-common/target/cygnus-common-1.7.0_SNAPSHOT-jar-with-dependencies.jar   ~/.m2/repository/com/telefonica/iot/cygnus-common/1.7.0

现在我在〜/ .m2 / repository / com / telefonica / iot / cygnus-common / 1.7.0目录下有两个罐子,cygnus-common-1.7.0_SNAPSHOT-jar-with-dependencies.jarcygnus-common-1.7.0.jar以及usr /包含它们的cygnus / plugins.d / cygnus / libext目录实际上我不知道这两个目录中是否存在两个罐子会产生问题。

我还认为cygnus-ngsi pom.xml文件可能是问题的一部分..这里是cygnus-ngsi的pom.xml:

    <project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.telefonica.iot</groupId>
    <artifactId>cygnus-ngsi</artifactId>
    <version>1.7.0_SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>cygnus-ngsi</name>
    <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <!-- Required for testing -->
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-all</artifactId>
            <version>1.9.5</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
        </dependency>
        <!-- Required by NGSIRestHandler -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore</artifactId>
            <version>4.3.1</version>
        </dependency>
        <!-- Required for logging -->
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
   </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.7.21</version>
        </dependency>
        <!-- Required by NameMapper -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.4</version>
        </dependency>
        <!-- Required by any agent -->
        <dependency>
            <groupId>com.telefonica.iot</groupId>
            <artifactId>cygnus-common</artifactId>
            <version>1.7.0_SNAPSHOT</version>
        </dependency>
    </dependencies>

    <build>
        <!-- reference: http://stackoverflow.com/questions/3697449/retrieve-version-from-maven-pom-xml-in-code -->
        <resources>
            <resource>
                <directory>src/main/resources</directory>
                <filtering>true</filtering>
            </resource>
        </resources>
        <pluginManagement>
            <plugins>
                <!-- Explicit Java version declaration in needed in some systems in which Maven seems not  -->
                <!-- being able to find itself the right version. Many systems don't need this declaration -->
                <!-- but it doesn't use to hurt anyway                                                     -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.5.1</version>
                    <configuration>
                        <source>1.7</source>
                        <target>1.7</target>
                        <compilerArgument></compilerArgument>
 <fork>true</fork>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-assembly-plugin</artifactId>
                    <version>2.6</version>
                    <configuration>
                        <descriptorRefs>
                            <descriptorRef>jar-with-dependencies</descriptorRef>
                        </descriptorRefs>
                    </configuration>
                </plugin>
                <!-- http://www.chrissearle.org/2009/09/05/Using_maven-exec-plugin_to_store_the_current_git_sha_in_a_build/ -->
                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>exec-maven-plugin</artifactId>
                    <version>1.5.0</version>
                    <executions>
                        <execution>
                            <phase>compile</phase>
                            <goals>
                                <goal>exec</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                        <executable>git</executable>
                        <arguments>
                            <argument>rev-parse</argument>
                            <argument>HEAD</argument>
                        </arguments>
                        <outputFile>target/classes/last_git_commit.txt</outputFile>
                    </configuration>
                </plugin>
   <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-site-plugin</artifactId>
                    <version>3.5.1</version>
                    <configuration>
                        <reportPlugins>
                            <plugin>
                                <groupId>org.codehaus.mojo</groupId>
                                <artifactId>cobertura-maven-plugin</artifactId>
                                <version>2.6</version>
                                <configuration>
                                    <aggregate>true</aggregate>
                                </configuration>
                            </plugin>
                            <plugin>
                                <groupId>org.apache.maven.plugins</groupId>
                                <artifactId>maven-surefire-report-plugin</artifactId>
                                <version>2.16</version>
                                <configuration>
                                    <aggregate>true</aggregate>
                                </configuration>
                            </plugin>
                            <plugin>
                                <groupId>org.apache.maven.plugins</groupId>
                                <artifactId>maven-checkstyle-plugin</artifactId>
                                <version>2.12</version>
                                <configuration>
                                    <configLocation>telefonica_checkstyle.xml</configLocation>
                                    <includeTestSourceDirectory>false</includeTestSourceDirectory>
                                    <failOnViolation>false</failOnViolation>
                                </configuration>
                            </plugin>
                            <plugin>
                                <groupId>org.apache.maven.plugins</groupId>
                                <artifactId>maven-javadoc-plugin</artifactId>
 <version>2.9</version>
                                <configuration>
                                    <aggregate>true</aggregate>
                                </configuration>
                            </plugin>
                        </reportPlugins>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

现在我想知道可能导致这种失败的原因。我想做的就是设置一个cygnus代理来使用ckansink,通过配置agent_.conf文件并提供必要的ckan参数然后启动cygnus。我知道除非天鹅座成功建造,否则这个过程不会完成

编辑:现在,我尝试通过运行

删除所有以前的天鹅座文件
sudo rpm -e -vv --allmatches --nodeps --noscripts --notriggers cygnus
sudo rpm -e -vv --allmatches --nodeps --noscripts --notriggers cygnus-ngsi

然后我使用rpm安装了cygnus-common和cygnus-ngsi 1.7.0 并且它们已成功安装。然后我在/usr/cygnus/conf/agent_1.conf文件中配置了必要的ckan参数。

现在的问题是我通过运行

启动天鹅座
/usr/cygnus/bin/cygnus-flume-ng agent --conf /usr/cygnus/conf/ -f /usr/cygnus/conf/agent_1.conf -n cygnusagent -Dflume.root.logger=INFO,console

我得到的这些日志只能通过侦听8081的jetty服务器结束,如下所示:

Info: Sourcing environment configuration script /usr/cygnus/conf/flume-env.sh
+ exec /usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin/java -Xmx20m -Dflume.root.logger=INFO,console -cp '/usr/cygnus/conf:/usr/cygnus/lib/*:/usr/cygnus/plugins.d/cygnus/lib/*:/usr/cygnus/plugins.d/cygnus/libext/*' -Djava.library.path= com.telefonica.iot.cygnus.nodes.CygnusApplication -f /usr/cygnus/conf/agent_1.conf -n cygnusagent
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/cygnus/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/cygnus/plugins.d/cygnus/lib/cygnus-ngsi-1.7.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/cygnus/plugins.d/cygnus/libext/cygnus-common-1.7.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
time=2017-04-18T16:34:31.961Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp= | op=main | msg=com.telefonica.iot.cygnus.nodes.CygnusApplication[166] : Starting Cygnus, version 1.7.0.UNKNOWN
time=2017-04-18T16:34:32.156Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=main | msg=com.telefonica.iot.cygnus.nodes.CygnusApplication[289] : Waiting for valid Flume components references...
time=2017-04-18T16:34:32.156Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=start | msg=org.apache.flume.node.PollingPropertiesFileConfigurationProvider[61] : Configuration provider starting
time=2017-04-18T16:34:32.159Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=run | msg=org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable[133] : Reloading configuration file:/usr/cygnus/conf/agent_1.conf
time=2017-04-18T16:34:32.182Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:dynamodb-sink
time=2017-04-18T16:34:32.182Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:hdfs-sink
time=2017-04-18T16:34:32.182Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:hdfs-sink
time=2017-04-18T16:34:32.182Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:sth-sink
time=2017-04-18T16:34:32.182Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.187Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:dynamodb-sink
time=2017-04-18T16:34:32.187Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.187Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:cartodb-sink
time=2017-04-18T16:34:32.188Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.188Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:cartodb-sink
time=2017-04-18T16:34:32.188Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2017-04-18T16:34:32.188Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mysql-sink
time=2017-04-18T16:34:32.190Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.191Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mysql-sink
time=2017-04-18T16:34:32.191Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:hdfs-sink
time=2017-04-18T16:34:32.191Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.191Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.191Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:hdfs-sink
time=2017-04-18T16:34:32.191Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[930] : Added sinks: hdfs-sink mysql-sink ckan-sink mongo-sink sth-sink kafka-sink dynamodb-sink postgresql-sink cartodb-sink Agent: cygnus-ngsi
time=2017-04-18T16:34:32.191Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.191Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:kafka-sink
time=2017-04-18T16:34:32.192Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:cartodb-sink
time=2017-04-18T16:34:32.192Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:hdfs-sink
time=2017-04-18T16:34:32.192Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:hdfs-sink
time=2017-04-18T16:34:32.192Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.192Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.192Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.192Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.192Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:hdfs-sink
time=2017-04-18T16:34:32.193Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:kafka-sink
time=2017-04-18T16:34:32.193Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.193Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.193Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.193Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:sth-sink
time=2017-04-18T16:34:32.193Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:dynamodb-sink
time=2017-04-18T16:34:32.193Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.194Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:dynamodb-sink
time=2017-04-18T16:34:32.194Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:postgresql-sink
time=2017-04-18T16:34:32.194Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:postgresql-sink
time=2017-04-18T16:34:32.195Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.195Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:ckan-sink
time=2017-04-18T16:34:32.198Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:mongo-sink
time=2017-04-18T16:34:32.220Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=validateConfiguration | msg=org.apache.flume.conf.FlumeConfiguration[140] : Post-validation flume configuration contains configuration for agents: [cygnus-ngsi]
time=2017-04-18T16:34:32.221Z | lvl=WARN | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=getConfiguration | msg=org.apache.flume.node.AbstractConfigurationProvider[138] : No configuration found for this host:cygnusagent
time=2017-04-18T16:34:32.224Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=startAllComponents | msg=org.apache.flume.node.Application[138] : Starting new configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
time=2017-04-18T16:34:33.156Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnusagent | op=main | msg=com.telefonica.iot.cygnus.nodes.CygnusApplication[301] : Starting a Jetty server listening on 0.0.0.0:8081 (Management Interface)

当我使用notification.sh测试文件从另一个终端测试天鹅座时,我得到连接拒绝。

[centos@orcb2 ~]$ ./notification.sh http://localhost:5050/notify
* About to connect() to localhost port 5050 (#0)
*   Trying ::1... Connection refused
*   Trying 127.0.0.1... Connection refused
* couldn't connect to host
* Closing connection #0
curl: (7) couldn't connect to host

虽然我认为配置文件应该是正确的 它们如下:

  • 这些是我的agent_1.conf文件源和ckan配置参数:

    =============================================
    source configuration
    
    cygnus-ngsi.sources.http-source.channels = hdfs-channel mysql-channel ckan-channel mongo-channel sth-channel kafka-channel dynamodb-channel postgresql-channel
    
    
    cygnus-ngsi.sources.http-source.type = org.apache.flume.source.http.HTTPSource
    
    cygnus-ngsi.sources.http-source.port = 5050
    
    cygnus-ngsi.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.NGSIRestHandler
    
    cygnus-ngsi.sources.http-source.handler.notification_target = /notify
    
    cygnus-ngsi.sources.http-source.handler.default_service = default
    
    cygnus-ngsi.sources.http-source.handler.default_service_path = /
    
    cygnus-ngsi.sources.http-source.interceptors = ts gi
    
    cygnus-ngsi.sources.http-source.interceptors.ts.type = timestamp
    
    cygnus-ngsi.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.NGSIGroupingInterceptor$Builder
    
    cygnus-ngsi.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf
    
    
    
       # ============================================
        NGSICKANSink configuration
    cygnus-ngsi.sinks.ckan-sink.channel = ckan-channel
    
    cygnus-ngsi.sinks.ckan-sink.type =com.telefonica.iot.cygnus.sinks.NGSICKANSink
    
    cygnus-ngsi.sinks.ckan-sink.enable_grouping = false
    
    cygnus-ngsi.sinks.ckan-sink.enable_name_mappings = false
    
    cygnus-ngsi.sinks.ckan-sink.enable_encoding = false
    
    cygnus-ngsi.sinks.ckan-sink.api_key = xxxxxxxx-xxxx-xxxxxx-xxxxx
    cygnus-ngsi.sinks.ckan-sink.ckan_host = http://130.206.120.212
    
    cygnus-ngsi.sinks.ckan-sink.ckan_port = 8080
    
    cygnus-ngsi.sinks.ckan-sink.ckan_viewer = recline_grid_view
    
    cygnus-ngsi.sinks.ckan-sink.orion_url = http://localhost:1026
    
    cygnus-ngsi.sinks.ckan-sink.attr_persistence = row
    
    cygnus-ngsi.sinks.ckan-sink.ssl = false
    
    cygnus-ngsi.sinks.ckan-sink.batch_size = 100
    
    cygnus-ngsi.sinks.ckan-sink.batch_timeout = 30
    
    cygnus-ngsi.sinks.ckan-sink.batch_ttl = 10
    
    cygnus-ngsi.sinks.ckan-sink.backend.max_conns = 500
    
    cygnus-ngsi.sinks.ckan-sink.backend.max_conns_per_route = 100
    

这是我的cygnus_instance_1.conf

CYGNUS_USER=cygnus

CONFIG_FOLDER=/usr/cygnus/conf

CONFIG_FILE=/usr/cygnus/conf/agent_1.conf

AGENT_NAME=cygnusagent

LOGFILE_NAME=cygnus.log

ADMIN_PORT=8081

POLLING_INTERVAL=30

那么,整个配置中是否缺少以及可以修复哪些内容以使httpsource接受我的通知请求?

2 个答案:

答案 0 :(得分:2)

从此命令:

cp target/cygnus-common-1.1.0-jar-with-dependencies.jar /usr/cygnus/plugins.d/cygnus/libext

您似乎已经构建了cygnus-common版本1.1.0(是不是?我的意思是,cygnus-common-1.1.0-jar-with-dependencies.jar目录中是否有target/文件?)

然而,在下一个命令中,您尝试安装了cygnus-common release 1.7.0_SNAPSHOT(带依赖项):

mvn install:install-file -Dfile=/usr/cygnus/plugins.d/cygnus/libext/cygnus-common-1.7.0_SNAPSHOT-jar-with-dependencies.jar -DgroupId=com.telefonica.iot -DartifactId=cygnus-common -Dversion=1.7.0 -Dpackaging=jar -DgeneratePom=false

因此上述命令之一是错误的:两者都必须引用相同的版本。

一个问题:一旦克隆了github repo,您​​是否按照文档中的说明签出了特定版本?

  

$ git clone https://github.com/telefonicaid/fiware-cygnus.git   
$ cd fiware-cygnus   
$ git checkout&lt; branch&gt;

这样的<branch>可能是master(最新开发,不推荐),1.7.0(最新官方发布),1.6.0 ...

因此,始终使用所有组件的特定版本,甚至使用适当的文档版本(例如,您发布的文档链接指的是版本1.3.0,而您尝试构建cygnus-common 1.1.0但安装1.7.0 ...)。

PS:有一种简单的方法可以安装最新版本(在撰写本文时,1.7.0):使用FIWARE repo中的RPM

编辑1:[修复安装问题并编辑了有关启动问题的问题]

如果你无法向Cygnus发送模拟通知,那是因为Cygnus没有正确启动。您可以在/var/log/cygnus/cygnus.log查看日志并获取更多详细信息。无论如何,对你的配置有一些评论:

关于agent_1.conf

  • 我缺少源,通道和接收器声明。另外,我错过了频道配置。为简单起见,我猜你有意识地省略了它。
  • 源渠道可以限制为ckan-channel

关于cygnus_instance_1.conf

  • AGENT_NAME必须与agent_1.conf中使用的匹配,即cygnus-ngsi

此外,安装后,您是否开始了Cygnus服务?只是为了确定:)

$ sudo service cygnus start

答案 1 :(得分:2)

编辑2 :我注意到cygnus_instance_1.conf中的默认命名是cygnusagent,同时在agent_1.conf中使用的是cygnus-ngsi,我已经解决了连接拒绝错误实例文件中的名称。现在我得到了成功的http响应,但是仍然没有在ckan中保存数据,因为我注意到天鹅日志中的一些错误(无论我是通过“service cygnus start”还是其他长启动命令启动cygnus)。

现在,这是成功的回应如下:

  [centos@orcb2 ~]$ ./notification.sh http://localhost:5050/notify
    * About to connect() to localhost port 5050 (#0)
    *   Trying ::1... connected
    * Connected to localhost (::1) port 5050 (#0)
    > POST /notify HTTP/1.1
    > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
    > Host: localhost:5050
    > Content-Type: application/json; charset=utf-8
    > Accept: application/json
    > Fiware-Service: gp
    > Fiware-ServicePath: /gp
    > Content-Length: 607
    > 
    < HTTP/1.1 200 OK
    < Transfer-Encoding: chunked
    < Server: Jetty(6.1.26)
    < 
    * Connection #0 to host localhost left intact
    * Closing connection #0

以下是关于启动天鹅座发送通知时新日志中的错误的重要部分:

Info: Sourcing environment configuration script /usr/cygnus/conf/flume-env.sh
Warning: JAVA_HOME is not set!
+ exec /usr/bin/java -Xmx20m -Dflume.log.file=cygnus.log -Duser.timezone=UTC -Dfile.encoding=UTF-8 -cp '/usr/cygnus/conf:/usr/cygnus/lib/*:/usr/cygnus/plugins.d/cygnus/lib/*:/usr/cygnus/plugins.d/cygnus/libext/*' -Djava.library.path= com.telefonica.iot.cygnus.nodes.CygnusApplication -p 8081 -f /usr/cygnus/conf/agent_1.conf -n cygnus-ngsi
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/cygnus/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/cygnus/plugins.d/cygnus/lib/cygnus-ngsi-1.7.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/cygnus/plugins.d/cygnus/libext/cygnus-common-1.7.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
time=2017-04-19T21:43:39.994Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp= | op=main | msg=com.telefonica.iot.cygnus.nodes.CygnusApplication[166] : Starting Cygnus, version 1.7.0.UNKNOWN
time=2017-04-19T21:43:40.136Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=main | msg=com.telefonica.iot.cygnus.nodes.CygnusApplication[289] : Waiting for valid Flume components references...
time=2017-04-19T21:43:40.139Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=org.apache.flume.node.PollingPropertiesFileConfigurationProvider[61] : Configuration provider starting
time=2017-04-19T21:43:40.141Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=run | msg=org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable[133] : Reloading configuration file:/usr/cygnus/conf/agent_1.conf
time=2017-04-19T21:43:40.166Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:dynamodb-sink
time=2017-04-19T21:43:40.166Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=addProperty | msg=org.apache.flume.conf.FlumeConfiguration$AgentConfiguration[1016] : Processing:hdfs-sink
/bin/bash: indent: command not found
.
.
.


   time=2017-04-19T21:43:40.364Z | lvl=ERROR | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSICartoDBSink[242] : [cartodb-sink] Error while creating the CartoDB persistence backend. Details: /usr/cygnus/conf/cartodb_keys.conf (No such file or directory)
time=2017-04-19T21:43:40.364Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSISink[344] : [cartodb-sink] Startup completed. Nevertheless, there are errors in the configuration, thus this sink will not run the expected logic
time=2017-04-19T21:43:40.378Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSISink[354] : [postgresql-sink] Startup completed
time=2017-04-19T21:43:40.386Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSIPostgreSQLSink[201] : [postgresql-sink] Startup completed
time=2017-04-19T21:43:40.379Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSISink[354] : [mongo-sink] Startup completed
time=2017-04-19T21:43:40.379Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSISink[354] : [sth-sink] Startup completed
time=2017-04-19T21:43:40.389Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=<init> | msg=com.telefonica.iot.cygnus.interceptors.CygnusGroupingRules[53] : Grouping rules read:
time=2017-04-19T21:43:40.403Z | lvl=WARN | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=<init> | msg=com.telefonica.iot.cygnus.interceptors.CygnusGroupingRules[61] : Grouping rules syntax has errors. Details: null
time=2017-04-19T21:43:40.405Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSISink[354] : [mysql-sink] Startup completed
time=2017-04-19T21:43:40.457Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=<init> | msg=com.telefonica.iot.cygnus.interceptors.CygnusGroupingRules[53] : Grouping rules read:
time=2017-04-19T21:43:40.457Z | lvl=WARN | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=<init> | msg=com.telefonica.iot.cygnus.interceptors.CygnusGroupingRules[61] : Grouping rules syntax has errors. Details: null
time=2017-04-19T21:43:40.744Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSISink[354] : [kafka-sink] Startup completed
time=2017-04-19T21:43:40.867Z | lvl=WARN | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=warn | msg=org.mortbay.log.Slf4jLog[76] : failed SocketConnector@0.0.0.0:5050: java.net.BindException: Address already in use (Bind failed)
time=2017-04-19T21:43:40.868Z | lvl=WARN | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=warn | msg=org.mortbay.log.Slf4jLog[76] : failed Server@15e1356d: java.net.BindException: Address already in use (Bind failed)
time=2017-04-19T21:43:40.868Z | lvl=ERROR | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=org.apache.flume.source.http.HTTPSource[154] : Error while starting HTTPSource. Exception follows.
java.net.BindException: Address already in use (Bind failed)
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
        at java.net.ServerSocket.bind(ServerSocket.java:376)
        at java.net.ServerSocket.<init>(ServerSocket.java:237)
        at org.mortbay.jetty.bio.SocketConnector.newServerSocket(SocketConnector.java:80)
        at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
        at org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
        at org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.Server.doStart(Server.java:235)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.flume.source.http.HTTPSource.start(HTTPSource.java:151)
        at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
        at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
time=2017-04-19T21:43:40.894Z | lvl=ERROR | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=run | msg=org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable[253] : Unable to start EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} } - Exception follows.
java.lang.RuntimeException: java.net.BindException: Address already in use (Bind failed)
        at com.google.common.base.Throwables.propagate(Throwables.java:156)
        at org.apache.flume.source.http.HTTPSource.start(HTTPSource.java:155)
        at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
        at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.BindException: Address already in use (Bind failed)
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
        at java.net.ServerSocket.bind(ServerSocket.java:376)
        at java.net.ServerSocket.<init>(ServerSocket.java:237)
        at org.mortbay.jetty.bio.SocketConnector.newServerSocket(SocketConnector.java:80)
        at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
        at org.mortbay.jetty.AbstractConnector.doStart(AbstractConnector.java:283)
        at org.mortbay.jetty.bio.SocketConnector.doStart(SocketConnector.java:147)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.mortbay.jetty.Server.doStart(Server.java:235)
        at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
        at org.apache.flume.source.http.HTTPSource.start(HTTPSource.java:151)
        ... 9 more


.
.
.


 time=2017-04-19T21:43:41.136Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=main | msg=com.telefonica.iot.cygnus.nodes.CygnusApplication[301] : Starting a Jetty server listening on 0.0.0.0:8081 (Management Interface)
time=2017-04-19T21:43:41.173Z | lvl=WARN | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=warn | msg=org.mortbay.log.Slf4jLog[76] : failed SelectChannelConnector@0.0.0.0:8081: java.net.BindException: Address already in use
/bin/bash: indent: command not found
time=2017-04-19T21:43:41.174Z | lvl=FATAL | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=run | msg=com.telefonica.iot.cygnus.http.JettyServer[90] : Fatal error running the Management Interface. Details=Address already in use
time=2017-04-19T21:43:41.446Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=start | msg=com.telefonica.iot.cygnus.sinks.NGSISink[354] : [dynamodb-sink] Startup completed
time=2017-04-19T21:43:42.169Z | lvl=ERROR | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=run | msg=com.telefonica.iot.cygnus.nodes.CygnusApplication$YAFS[457] : Thread found not alive, exiting Cygnus. ID=41, name=Thread-21
Starting an ordered shutdown of Cygnus
Stopping sources
Stopping http-source (lyfecycle state=IDLE)
time=2017-04-19T21:43:42.173Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[139] : Component type: SOURCE, name: http-source stopped
time=2017-04-19T21:43:42.173Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[145] : Shutdown Metric for type: SOURCE, name: http-source. source.start.time == 0
time=2017-04-19T21:43:42.173Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[151] : Shutdown Metric for type: SOURCE, name: http-source. source.stop.time == 1492638222173
time=2017-04-19T21:43:42.174Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: SOURCE, name: http-source. src.append-batch.accepted == 0
time=2017-04-19T21:43:42.179Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: SOURCE, name: http-source. src.append-batch.received == 0
time=2017-04-19T21:43:42.180Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: SOURCE, name: http-source. src.append.accepted == 0
time=2017-04-19T21:43:42.180Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: SOURCE, name: http-source. src.append.received == 0
time=2017-04-19T21:43:42.180Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: SOURCE, name: http-source. src.events.accepted == 0
time=2017-04-19T21:43:42.180Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: SOURCE, name: http-source. src.events.received == 0
time=2017-04-19T21:43:42.180Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: SOURCE, name: http-source. src.open-connection.count == 0
time=2017-04-19T21:43:42.181Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.source.http.HTTPSource[172] : Http source http-source stopped. Metrics: SOURCE:http-source{src.events.accepted=0, src.events.received=0, src.append.accepted=0, src.append-batch.accepted=0, src.open-connection.count=0, src.append-batch.received=0, src.append.received=0}
All the channels are empty
Stopping channels
.
.
.

Stopping ckan-channel (lyfecycle state=START)
time=2017-04-19T21:44:10.482Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[139] : Component type: CHANNEL, name: ckan-channel stopped
time=2017-04-19T21:44:10.482Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[145] : Shutdown Metric for type: CHANNEL, name: ckan-channel. channel.start.time == 1492638220354
time=2017-04-19T21:44:10.482Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[151] : Shutdown Metric for type: CHANNEL, name: ckan-channel. channel.stop.time == 1492638250482
time=2017-04-19T21:44:10.482Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: CHANNEL, name: ckan-channel. channel.capacity == 1000
time=2017-04-19T21:44:10.483Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: CHANNEL, name: ckan-channel. channel.current.size == 0
time=2017-04-19T21:44:10.483Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: CHANNEL, name: ckan-channel. channel.event.put.attempt == 0
time=2017-04-19T21:44:10.483Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: CHANNEL, name: ckan-channel. channel.event.put.success == 0
time=2017-04-19T21:44:10.483Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: CHANNEL, name: ckan-channel. channel.event.take.attempt == 5
time=2017-04-19T21:44:10.483Z | lvl=INFO | corr= | trans= | srv= | subsrv= | comp=cygnus-ngsi | op=stop | msg=org.apache.flume.instrumentation.MonitoredCounterGroup[167] : Shutdown Metric for type: CHANNEL, name: ckan-channel. channel.event.take.success == 0
.
.
.
Stopping sinks
Stopping ckan-sink (lyfecycle state=START)
Stopping postgresql-sink (lyfecycle state=START)
Stopping dynamodb-sink (lyfecycle state=START)
Stopping kafka-sink (lyfecycle state=START)
Stopping cartodb-sink (lyfecycle state=START)
Stopping hdfs-sink (lyfecycle state=START)
Stopping mongo-sink (lyfecycle state=START)
Stopping mysql-sink (lyfecycle state=START)
Stopping sth-sink (lyfecycle state=START)

这些是agent.conf文件中的通道,源和接收器定义

agent.sources = seqGenSrc
agent.channels = memoryChannel
agent.sinks = loggerSink

agent.sources.seqGenSrc.type = seq

agent.sources.seqGenSrc.channels = memoryChannel

agent.sinks.loggerSink.type = logger

agent.sinks.loggerSink.channel = memoryChannel

agent.channels.memoryChannel.type = memory


agent.channels.memoryChannel.capacity = 100

这就是ckan频道配置

cygnus-ngsi.channels.ckan-channel.type = memory
cygnus-ngsi.channels.ckan-channel.capacity = 1000
cygnus-ngsi.channels.ckan-channel.transactionCapacity = 100

关于ckan方面,我想知道Ckan中是否还有其他程序,而不是创建帐户以持久化ngsi事件。我是否必须手动创建组织和数据集及其数据存储区以接收ngsi事件?或是将fiware服务和服务路径映射到组织和数据集,并在发送ngsi事件时自动创建其数据存储和资源?我只是想确定我是否理解这一点。

现在,您认为我将如何解决这些错误?