我构建了德鲁伊，并在其中加载批量数据时出错，如下所示：
2016-10-19T07:06:08,656 INFO [task-runner-0-priority-0] org.apache.hadoop.mapreduce.Job - Counters: 12
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=3
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=13910
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=13910
Total vcore-seconds taken by all map tasks=13910
Total megabyte-seconds taken by all map tasks=14243840
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
2016-10-19T07:06:08,657 ERROR [task-runner-0-priority-0] io.druid.indexer.DetermineHashedPartitionsJob - Job failed: job_1476065812127_0012
2016-10-19T07:06:08,657 INFO [task-runner-0-priority-0] io.druid.indexer.JobHelper - Deleting path[var/druid/hadoop-tmp/wikiticker/2016-10-19T070532.438Z/9d778c51830e449f831a3966a679df11]
2016-10-19T07:06:08,682 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_hadoop_wikiticker_2016-10-19T07:05:32.450Z, type=index_hadoop, dataSource=wikiticker}]
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.1.jar:?]
at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:204) ~[druid-indexing-service-0.9.1.1.jar:0.9.1.1]
at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:175) ~[druid-indexing-service-0.9.1.1.jar:0.9.1.1]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.9.1.1.jar:0.9.1.1]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.9.1.1.jar:0.9.1.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_79]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_79]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_79]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_79]
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_79]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_79]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_79]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_79]
at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:201) ~[druid-indexing-service-0.9.1.1.jar:0.9.1.1]
... 7 more
Caused by: com.metamx.common.ISE: Job[class io.druid.indexer.DetermineHashedPartitionsJob] failed!
at io.druid.indexer.JobHelper.runJobs(JobHelper.java:343) ~[druid-indexing-hadoop-0.9.1.1.jar:0.9.1.1]
at io.druid.indexer.HadoopDruidDetermineConfigurationJob.run(HadoopDruidDetermineConfigurationJob.java:91) ~[druid-indexing-hadoop-0.9.1.1.jar:0.9.1.1]
at io.druid.indexing.common.task.HadoopIndexTask$HadoopDetermineConfigInnerProcessing.runTask(HadoopIndexTask.java:291) ~[druid-indexing-service-0.9.1.1.jar:0.9.1.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_79]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_79]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_79]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_79]
at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:201) ~[druid-indexing-service-0.9.1.1.jar:0.9.1.1]
... 7 more
2016-10-19T07:06:08,687 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_hadoop_wikiticker_2016-10-19T07:05:32.450Z] status changed to [FAILED].
2016-10-19T07:06:08,688 INFO [task-runner-0-priority-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
"id" : "index_hadoop_wikiticker_2016-10-19T07:05:32.450Z",
"status" : "FAILED",
"duration" : 33220
}
然后根据official solution，我做了一些操作，但也没什么用处：
编辑Druid的pom.xml依赖项以匹配我的Hadoop版本中的Jackson版本并重新编译德鲁伊
Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.introspect.AnnotatedMember.annotations()Ljava/lang/Iterable;
使用maven-shade-plugin制作胖罐装德鲁伊服务-0.9.1.1-selfcontained.jar
java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath ~/druid-0.9.1.1/tmp/druid-services-0.9.1.1-selfcontained.jar:~/druid-0.9.1.1/conf/druid/_common/:~/jackson-* -Ddruid.extensions.loadList=[] io.druid.cli.Main index hadoop --coordinate "org.apache.hadoop:hadoop-client:2.6.0-cdh5.7.1" ../quickstart/wikiticker-index.json
执行这个共同点，我得到了：
INFO: An exception was caught and reported. Message: java.lang.NullPointerException
java.lang.NullPointerException
at io.druid.cli.CliInternalHadoopIndexer$1.configure(CliInternalHadoopIndexer.java:95)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:230)
at com.google.inject.spi.Elements.getElements(Elements.java:103)
at com.google.inject.spi.Elements.getElements(Elements.java:94)
at com.google.inject.util.Modules$RealOverriddenModuleBuilder$1.configure(Modules.java:173)
at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:230)
at com.google.inject.spi.Elements.getElements(Elements.java:103)
at com.google.inject.spi.Elements.getElements(Elements.java:94)
at com.google.inject.util.Modules$RealOverriddenModuleBuilder$1.configure(Modules.java:153)
at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:230)
at com.google.inject.spi.Elements.getElements(Elements.java:103)
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:136)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
at com.google.inject.Guice.createInjector(Guice.java:96)
at com.google.inject.Guice.createInjector(Guice.java:73)
at com.google.inject.Guice.createInjector(Guice.java:62)
at io.druid.initialization.Initialization.makeInjectorWithModules(Initialization.java:367)
at io.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:60)
at io.druid.cli.CliInternalHadoopIndexer.run(CliInternalHadoopIndexer.java:108)
at io.druid.cli.Main.main(Main.java:105)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at io.druid.cli.CliHadoopIndexer.run(CliHadoopIndexer.java:115)
at io.druid.cli.Main.main(Main.java:105)
2016-10-20T05:05:52,401 ERROR [main] io.druid.cli.CliHadoopIndexer - failure!!!!
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_79]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_79]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_79]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_79]
at io.druid.cli.CliHadoopIndexer.run(CliHadoopIndexer.java:115) [druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at io.druid.cli.Main.main(Main.java:105) [druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
Caused by: com.google.inject.CreationException: Guice creation errors:
1) An exception was caught and reported. Message: null
at com.google.inject.util.Modules$RealOverriddenModuleBuilder$1.configure(Modules.java:173)
2) Binding to null instances is not allowed. Use toProvider(Providers.of(null)) if this is your intended behaviour.
at io.druid.cli.CliInternalHadoopIndexer$1.configure(CliInternalHadoopIndexer.java:93)
3) Could not find a suitable constructor in io.druid.metadata.MetadataStorageTablesConfig. Classes must have either one (and only one) constructor annotated with @Inject or a zero-argument constructor that is not private.
at io.druid.metadata.MetadataStorageTablesConfig.class(MetadataStorageTablesConfig.java:35)
at io.druid.cli.CliInternalHadoopIndexer$1.configure(CliInternalHadoopIndexer.java:95)
3 errors
at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:448) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:155) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:107) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.Guice.createInjector(Guice.java:96) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.Guice.createInjector(Guice.java:73) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.Guice.createInjector(Guice.java:62) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at io.druid.initialization.Initialization.makeInjectorWithModules(Initialization.java:367) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at io.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:60) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at io.druid.cli.CliInternalHadoopIndexer.run(CliInternalHadoopIndexer.java:108) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at io.druid.cli.Main.main(Main.java:105) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
... 6 more
Caused by: java.lang.NullPointerException
at io.druid.cli.CliInternalHadoopIndexer$1.configure(CliInternalHadoopIndexer.java:95) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:230) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.spi.Elements.getElements(Elements.java:103) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.spi.Elements.getElements(Elements.java:94) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.util.Modules$RealOverriddenModuleBuilder$1.configure(Modules.java:173) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.AbstractModule.configure(AbstractModule.java:62) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:230) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.spi.Elements.getElements(Elements.java:103) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.spi.Elements.getElements(Elements.java:94) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.util.Modules$RealOverriddenModuleBuilder$1.configure(Modules.java:153) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.AbstractModule.configure(AbstractModule.java:62) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:230) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.spi.Elements.getElements(Elements.java:103) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:136) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.Guice.createInjector(Guice.java:96) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.Guice.createInjector(Guice.java:73) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at com.google.inject.Guice.createInjector(Guice.java:62) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at io.druid.initialization.Initialization.makeInjectorWithModules(Initialization.java:367) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at io.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:60) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at io.druid.cli.CliInternalHadoopIndexer.run(CliInternalHadoopIndexer.java:108) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
at io.druid.cli.Main.main(Main.java:105) ~[druid-services-0.9.1.1-selfcontained.jar:0.9.1.1]
... 6 more
有没有人有同样的问题。有人可以帮忙吗？
答案 0 :(得分：1)
我能够运行索引编制工作。不需要遮蔽的罐子。以下是步骤。
1）下载与您的CDH匹配的Hadoop客户端。在druid发行目录中运行此命令。它会将Hadoop客户端下载到hadoop-dependencies目录。
./bin/pull-deps -r https://repository.cloudera.com/content/repositories/releases/ -h org.apache.hadoop:hadoop-client:2.6.0-mr1-cdh5.7.1
2）在middleManger配置中设置客户端版本。
hadoopDependencyCoordinates=["org.apache.hadoop:hadoop-client:2.6.0-mr1-cdh5.7.1"]
3）我使用索引服务来运行Hadoop作业。创建JSON，并像这样放置jobProperties。
"jobProperties": {
"mapreduce.job.classloader": "true",
"mapreduce.job.classloader.system.classes": "-javax.validation.,java.,javax.,org.apache.commons.logging.,org.apache.log4j.,org.apache.hadoop.,org.xerial.snappy."
}
如果您仍然看到一些错误并且知道它存在于CDH和Druid中，则根据需要将类添加到列表中。
4）HTTP将JSON发布到Overlord