Question

I am using Leiningen 2.5.3 on Java 1.7.0_79 OpenJDK 64-Bit Server VM.

我想使用lein uberjar排除storm-core jar。下面是我的project.clj

(defproject kafka2hdfs "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [org.apache.storm/storm-kafka "0.9.5"]
                 [org.apache.storm/storm-hdfs "0.9.5"]
                 [org.apache.kafka/kafka_2.10 "0.8.2.1"]]
  :plugins [[cider/cider-nrepl "0.10.0-SNAPSHOT"]]
  :target-path "target/%s"
  :dev-dependencies [[org.apache.storm/storm-core "0.9.5"]]
  :main kafka2hdfs.core
  :aot [kafka2hdfs.core])

从lein uberjar获取错误输出，如何解决这个问题？

$ lein uberjar
Compiling kafka2hdfs.core
java.lang.ClassNotFoundException: backtype.storm.StormSubmitter, compiling:(core.clj:1:1)
Exception in thread "main" java.lang.ClassNotFoundException: backtype.storm.StormSubmitter, compiling:(core.clj:1:1)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3463)

我的core.clj文件

(ns kafka2hdfs.core
  (:import [backtype.storm StormSubmitter LocalCluster spout.SchemeAsMultiScheme]
           [storm.kafka ZkHosts SpoutConfig StringScheme KafkaSpout]
           [org.apache.storm.hdfs.bolt HdfsBolt]
           [org.apache.storm.hdfs.bolt.format DefaultFileNameFormat DelimitedRecordFormat]
           [org.apache.storm.hdfs.bolt.sync CountSyncPolicy]
           [org.apache.storm.hdfs.bolt.rotation TimedRotationPolicy]
           )
  (:use [backtype.storm clojure config]) ;; for (topology ...)
  (:gen-class))

(defn mk-topology []
   ;; ......
  ）

    (topology
     {"kafka-reader" (spout-spec kafka-reader :p 2)}
     {"hdfs-writer" (bolt-spec {"kafka-reader" :shuffle} hdfs-writer :p 1)})
    ))

(defn submit-topology! [name]
  (StormSubmitter/submitTopology
   name
   {TOPOLOGY-DEBUG true
    TOPOLOGY-WORKERS 3}
   (mk-topology)))

(defn -main
  "a simple topology demo read from kafka ans write to hdfs"
  [& args]
  (submit-topology! "kafka2hdfs-topic-bigdata-obd"))

几个小时后，我发现如果我在project.clj中注释掉:aot [kafka2hdfs.core]，它可以正常输出警告信息：

$ lein do clean, uberjar
Warning: The Main-Class specified does not exist within the jar. It may not be executable as expected. A gen-class directive may be missing in the namespace which contains the main method.

我是lein的新手：aot，为什么？

Answer 1

我真的不明白了解如何排除jar会解决ClassNotFoundException，但你可以通过

排除leiningen中的传递依赖性

[org.apache.storm/storm-kafka "0.9.5" :exclusions [org.apache.storm/storm-core]]

看起来你错误的原因是缺少backtype/storm jar。也许您需要:import来自org.apache.storm而不是backtype.storm？

Answer 2

要从包中排除jar文件，需要了解lein project.clj将用于生成maven pom.xml。使用200 milliseconds命令生成pom.xml。在我的例子中，我只需要将提供的范围应用于storm-core依赖，并使用：exclusions来生成依赖项排除。所以我的project.clj看起来像这样

lein pom

使用lein pom生成pom.xml

(defproject kafka2hdfs "0.1.0-SNAPSHOT"
  :description "demo to show read from kafka and write to hdfs"
  :url "http://blog.csdn.net/csfreebird"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.5.1"]
                 [org.apache.kafka/kafka_2.10 "0.8.2.1"
                  :exclusions [[org.apache.zookeeper/zookeeper]
                               [log4j/log4j]
                               [slf4j-api/org.slf4j]]
                  ]
                 [org.apache.storm/storm-kafka "0.9.5"]
                 [org.apache.storm/storm-hdfs "0.9.5"]]
  :main kafka2hdfs.core
  :aot [kafka2hdfs.core]
  :profiles {:provided {:dependencies [[org.apache.storm/storm-core "0.9.5"
                                        :exclusions [[org.slf4j/log4j-over-slf4j]
                                                     [org.slf4j/slf4j-api]
                                                     [logback-classic/ch.qos.logback]]
                                        ]]}}
  :plugins [[cider/cider-nrepl "0.10.0-SNAPSHOT"]]
  :target-path "target/%s")

这是我的答案！未找到的类也解决了。

如何在clojure project.clj中排除storm-core jar

2 个答案: