我正在尝试将数据从PostgreSQL提取到Druid。索引任务已创建,并且无法引发NullPointerExcpetion。
摄入规范
{
"type": "index",
"spec": {
"dataSchema": {
"dataSource": "dataset_new10"
},
"ioConfig": {
"type": "index",
"firehose": {
"type": "sql",
"database": {
"type": "postgresql",
"connectorConfig": {
"connectURI": "jdbc:postgresql://ISVDRDBILXP1/testdb",
"user": "druid",
"password": "druid"
}
},
"sqls": [
"SELECT * FROM public.testtable"
]
}
}
}
}
这是我的堆栈跟踪:
2019-07-10T12:59:22,853 INFO [task-runner-0-priority-0] org.apache.druid.data.input.impl.prefetch.CacheManager - Object[SELECT * FROM public.testtable] is cached. Current cached bytes is [188]
2019-07-10T12:59:22,858 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.common.task.IndexTask - Encountered exception in DETERMINE_PARTITIONS.
java.lang.NullPointerException
at org.apache.druid.data.input.impl.SqlFirehose.nextRow(SqlFirehose.java:68) ~[druid-core-0.15.0-incubating.jar:0.15.0-incubating]
at org.apache.druid.indexing.common.task.IndexTask.collectIntervalsAndShardSpecs(IndexTask.java:744) ~[druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
at org.apache.druid.indexing.common.task.IndexTask.createShardSpecsFromInput(IndexTask.java:671) ~[druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
at org.apache.druid.indexing.common.task.IndexTask.determineShardSpecs(IndexTask.java:606) ~[druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
at org.apache.druid.indexing.common.task.IndexTask.run(IndexTask.java:437) [druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:419) [druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:391) [druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
答案 0 :(得分:0)
在 Druid 0.20.1 中,我修复了这个错误,添加了一些 JSON 标签:
{
"type": "index_parallel",
"spec": {
"dataSchema": {
"dataSource": "test-ingestion-postgresql",
"timestampSpec": {
"format": "iso",
"column": "ts",
"missingValue": "2021-05-05T00:00:00.000"
},
"dimensionsSpec": {
"dimensions": [
"column1",
"column2",
"column3"
]
}
},
"ioConfig": {
"type": "index_parallel",
"inputSource": {
"type": "sql",
"database": {
"type": "postgresql",
"connectorConfig": {
"connectURI": "jdbc:postgresql://host:5432/db_name",
"user": "user",
"password": "password"
}
},
"sqls": [
"SELECT column1, column2, column3 FROM schema.table"
]
}
},
"tuningConfig": {
"type": "index_parallel",
"partitionsSpec": {
"type": "dynamic"
}
}
}
}