列' nextfetchdate'的StormCrawler SQL错误

时间:2018-05-23 13:33:36

标签: web-crawler stormcrawler

我的设置与this相同。当我在爬网模式下运行crawler时,出现以下错误:

  

[Thread-130-status-executor [109 109]] ERROR c.d.s.p.AbstractStatusUpdaterBolt - 存储时捕获异常   com.mysql.jdbc.MysqlDataTruncation:数据截断:日期时间值不正确:' 2099-12-31 00:00:00'对于列' nextfetchdate'在第1行           在com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3964)〜[stromcrawler-1.0-SNAPSHOT.jar:?]           在com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3902)〜[stromcrawler-1.0-SNAPSHOT.jar:?]           在com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2526)〜[stromcrawler-1.0-SNAPSHOT.jar:?]           在com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2673)〜[stromcrawler-1.0-SNAPSHOT.jar:?]           在com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549)〜[stromcrawler-1.0-SNAPSHOT.jar:?]           在com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)〜[stromcrawler-1.0-SNAPSHOT.jar:?]           在com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1192)〜[stromcrawler-1.0-SNAPSHOT.jar:?]           在com.digitalpebble.stormcrawler.sql.StatusUpdaterBolt.store(StatusUpdaterBolt.java:132)〜[stromcrawler-1.0-SNAPSHOT.jar:?]           在com.digitalpebble.stormcrawler.persistence.AbstractStatusUpdaterBolt.execute(AbstractStatusUpdaterBolt.java:196)[stromcrawler-1.0-SNAPSHOT.jar:?]           在org.apache.storm.daemon.executor $ fn__5043 $ tuple_action_fn__5045.invoke(executor.clj:739)[storm-core-1.2.1.jar:1.2.1]           在org.apache.storm.daemon.executor $ mk_task_receiver $ fn__4964.invoke(executor.clj:468)[storm-core-1.2.1.jar:1.2.1]           在org.apache.storm.disruptor $ clojure_handler $ reify__4475.onEvent(disruptor.clj:41)[storm-core-1.2.1.jar:1.2.1]           在org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)[storm-core-1.2.1.jar:1.2.1]           在org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)[storm-core-1.2.1.jar:1.2.1]           at org.apache.storm.disruptor $ consume_batch_when_available.invoke(disruptor.clj:74)[storm-core-1.2.1.jar:1.2.1]           在org.apache.storm.daemon.executor $ fn__5043 $ fn__5056 $ fn__5109.invoke(executor.clj:861)[storm-core-1.2.1.jar:1.2.1]           at org.apache.storm.util $ async_loop $ fn__557.invoke(util.clj:484)[storm-core-1.2.1.jar:1.2.1]           在clojure.lang.AFn.run(AFn.java:22)[clojure-1.7.0.jar:?]           在java.lang.Thread.run(Thread.java:745)[?:1.8.0_121]

请建议在哪里修理它

1 个答案:

答案 0 :(得分:0)

这是由于fetch interval for errors已被设置为-1,这意味着“永远不会重新访问”。这实际上被DefaultScheduler翻译成了一个很久以前的日期。不清楚为什么mysql会呛到它。您可以尝试设置更合理的值,例如43200一个月,看看是否有效。