Question

我正在用Java创建一个flink系统来读写MongoDB。我在JDK 1.8.0_181中使用mongo-hadoop-core核心版本1.3.2和Apache flink 1.6。

当我在Eclipse中运行源代码时，一切都可以成功运行，但是当我使用flink命令运行JAR文件时，总是会收到此错误。

这是我的pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.flink</groupId>
<artifactId>test-mongodb-2</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.0.0</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <artifactSet>
                            <excludes>
                                <exclude>com.google.code.findbugs:jsr305</exclude>
                                <exclude>org.slf4j:*</exclude>
                                <exclude>log4j:*</exclude>
                            </excludes>
                        </artifactSet>
                        <filters>
                            <filter>
                                <!-- Do not copy the signatures in the META-INF folder.
                                Otherwise, this might cause SecurityExceptions when using the JAR. -->
                                <artifact>*:*</artifact>
                                <excludes>
                                    <exclude>META-INF/*.SF</exclude>
                                    <exclude>META-INF/*.DSA</exclude>
                                    <exclude>META-INF/*.RSA</exclude>
                                </excludes>
                            </filter>
                        </filters>
                        <transformers>
                            <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                <mainClass>my.programs.main.clazz</mainClass>
                            </transformer>
                        </transformers>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

<dependencies>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-java</artifactId>
      <version>1.6.0</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-streaming-java_2.11</artifactId>
      <version>1.6.0</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-core</artifactId>
        <version>1.6.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-clients_2.11</artifactId>
      <version>1.6.0</version>
    </dependency>
    <dependency>
        <groupId>org.mongodb.mongo-hadoop</groupId>
        <artifactId>mongo-hadoop-core</artifactId>
        <version>1.3.2</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-hadoop-compatibility_2.11</artifactId>
        <version>1.6.0</version>
    </dependency>
</dependencies>
</project>

但是我将mongo-hadoop-core版本更改为1.3.2后，会给出不同的错误

java.lang.RuntimeException: Could not look up the main(String[]) method from the class MongoDBExample: org/apache/flink/api/java/hadoop/mapred/HadoopInputFormat
at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:499)
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:218)
at org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
at org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:856)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:206)
at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1044)
at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1120)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1120)
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/api/java/hadoop/mapred/HadoopInputFormat
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at org.apache.flink.client.program.PackagedProgram.hasMainMethod(PackagedProgram.java:493)
... 11 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.api.java.hadoop.mapred.HadoopInputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 17 more

我在做什么错了？

Answer 1

我在pom.xml中添加了mongo-hadoop-core-2.0.2.jar，并在其中找到了该类。

请尝试CNTR + SHIFT + T并检查您的类路径中是否存在该类。

java.lang.ClassNotFoundException：com.mongodb.hadoop.mapred.MongoInputFormat在运行Apache Flink时

1 个答案: