我正在尝试运行一些spark应用。问题是maven似乎在其本地缓存仓库中发现了一些软件包，但是当它尝试加载它们时，却找不到它们，实际上某些软件包似乎存在但在它们意义上是不完整的不包含任何罐子。
这是日志：
com.databricks#spark-avro_2.10 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
confs: [default]
found com.databricks#spark-avro_2.10;1.0.0 in local-m2-cache
found org.apache.avro#avro;1.7.6 in local-m2-cache
found org.codehaus.jackson#jackson-core-asl;1.9.13 in local-m2-cache
found org.codehaus.jackson#jackson-mapper-asl;1.9.13 in local-m2-cache
found com.thoughtworks.paranamer#paranamer;2.3 in local-m2-cache
found org.xerial.snappy#snappy-java;1.0.5 in local-m2-cache
found org.apache.commons#commons-compress;1.4.1 in local-m2-cache
found org.tukaani#xz;1.0 in local-m2-cache
found org.slf4j#slf4j-api;1.6.4 in local-m2-cache
:: resolution report :: resolve 484ms :: artifacts dl 22ms
:: modules in use:
com.databricks#spark-avro_2.10;1.0.0 from local-m2-cache in [default]
com.thoughtworks.paranamer#paranamer;2.3 from local-m2-cache in [default]
org.apache.avro#avro;1.7.6 from local-m2-cache in [default]
org.apache.commons#commons-compress;1.4.1 from local-m2-cache in [default]
org.codehaus.jackson#jackson-core-asl;1.9.13 from local-m2-cache in [default]
org.codehaus.jackson#jackson-mapper-asl;1.9.13 from local-m2-cache in [default]
org.slf4j#slf4j-api;1.6.4 from local-m2-cache in [default]
org.tukaani#xz;1.0 from local-m2-cache in [default]
org.xerial.snappy#snappy-java;1.0.5 from local-m2-cache in [default]
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 9 | 0 | 0 | 0 || 9 | 0 |
---------------------------------------------------------------------
:: problems summary ::
:::: WARNINGS
[NOT FOUND ] org.xerial.snappy#snappy-java;1.0.5!snappy-java.jar(bundle) (1ms)
==== local-m2-cache: tried
file:/Users/someuser/.m2/repository/org/xerial/snappy/snappy-java/1.0.5/snappy-java-1.0.5.jar
[NOT FOUND ] org.slf4j#slf4j-api;1.6.4!slf4j-api.jar (1ms)
==== local-m2-cache: tried
file:/Users/someuser/.m2/repository/org/slf4j/slf4j-api/1.6.4/slf4j-api-1.6.4.jar
::::::::::::::::::::::::::::::::::::::::::::::
:: FAILED DOWNLOADS ::
:: ^ see resolution messages for details ^ ::
::::::::::::::::::::::::::::::::::::::::::::::
:: org.xerial.snappy#snappy-java;1.0.5!snappy-java.jar(bundle)
:: org.slf4j#slf4j-api;1.6.4!slf4j-api.jar
::::::::::::::::::::::::::::::::::::::::::::::
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
Exception in thread "main" java.lang.RuntimeException: [download failed: org.xerial.snappy#snappy-java;1.0.5!snappy-java.jar(bundle), download failed: org.slf4j#slf4j-api;1.6.4!slf4j-api.jar]
at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:995)
at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:263)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:145)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
我尝试删除.m2/repository文件夹然后再次运行maven，但它没有改变任何内容。
编辑：这是我的pom文件
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>spark-ontology</groupId>
<artifactId>spark-filter</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.4.1</version>
</dependency>
<dependency>
<groupId>com.databricks</groupId>
<artifactId>spark-avro_2.10</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.10</artifactId>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.7.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-common</artifactId>
<version>2.7.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-yarn-api</artifactId>
<version>2.7.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-common</artifactId>
<version>2.7.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-jobclient</artifactId>
<version>2.7.0</version>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>1.7.7</version>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-mapred</artifactId>
<version>1.7.7</version>
</dependency>
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.4.5</version>
</dependency>
</dependencies>
<properties>
<java.version>1.7</java.version>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
答案 0 :(得分：1)
尝试添加以下依赖项 -
<dependency>
<groupId>org.xerial.snappy</groupId>
<artifactId>snappy-java</artifactId>
<version>1.0.5</version>
</dependency>
Maven在准备构建时获取所有必需的依赖项。如果找不到依赖项，它将抛出一个错误，但除非特别告诉它需要什么依赖项，否则它无法解决它们。
在这种情况下，看起来其中一个命名的依赖项需要xerial 1.0.5，因此除非将其作为显式命名的依赖项提供，否则不会编译。
您需要运行'mvn clean package'以强制重新解析依赖项。 在运行“clean package”之前，您可能还需要从.m2 repo中的xerial文件夹中删除1.0.5目录。 有时，maven会看到目录（无论是否在其中），而不是尝试从中央仓库解析依赖。
自从我使用maven以来已经有好几次了，但我确实记得在网络连接失败的情况下发生了这种情况，而maven试图从回购中拾取一个jar，使.m2处于不一致的状态。