Question

从API下载数据时遇到一些问题。所以有API URL，如果我在浏览器中手动打开，我会以适当的XML格式向我提供XML文件。但我不想下载文件，保存然后加载它，但我希望直接获取数据并存储在表中。我正在使用Kettle在转换中使用“从XML获取数据”来加载此数据。我所做的只是提供指向该URL的链接并将输出存储在数据库表中。但在这种情况下，有时URL / API响应很慢，如果第一次尝试页面加载失败，我的任务就会出错。

我正在寻找，如果可能的话 -

延长加载API的超时时间
对系统进行重试，以便即使第一次失败也可以在5秒的间隔后再次尝试说。有没有办法做到这一点，而不是批量运行水壶，然后批量添加循环间隔时间而不是水壶？

日志错误 -

2014/08/08 05:30:04 - Purchase Sat ETL - Loading transformation from
XML file [file:///C:/Scripts/Kettle/Purchase Sat ETL.ktr] 2014/08/08
05:30:04 - Purchase Sat ETL - Dispatching started for transformation
[Purchase Sat ETL] 2014/08/08 05:30:05 - Table output.0 - Connected to
database [nav_st_copa_site_intercept] (commit=1000) 2014/08/08
05:30:05 - Generate Rows.0 - Finished processing (I=0, O=0, R=0, W=1,
U=0, E=0) 2014/08/08 05:30:05 - Set field value to a constant.0 -
Finished processing (I=0, O=0, R=1, W=1, U=0, E=0) 2014/08/08 05:32:08
- Get data from XML.0 - ERROR (version 5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) : Unexpected Error :
org.pentaho.di.core.exception.KettleException:  2014/08/08 05:32:08 -
Get data from XML.0 - org.dom4j.DocumentException: Server returned
HTTP response code: 500 for URL: <URL>StartDate=2014-07-30%2001:44:46
Nested exception: Server returned HTTP response code: 500 for URL:
<URL>StartDate=2014-07-30%2001:44:46 2014/08/08 05:32:08 - Get data
from XML.0 - Server returned HTTP response code: 500 for URL:
<URL>StartDate=2014-07-30%2001:44:46 Nested exception: Server returned
HTTP response code: 500 for URL: <URL>StartDate=2014-07-30%2001:44:46
2014/08/08 05:32:08 - Get data from XML.0 - ERROR (version
5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) : org.pentaho.di.core.exception.KettleException:  2014/08/08 05:32:08 -
Get data from XML.0 - org.dom4j.DocumentException: Server returned
HTTP response code: 500 for URL: <URL>StartDate=2014-07-30%2001:44:46
Nested exception: Server returned HTTP response code: 500 for URL:
<URL>StartDate=2014-07-30%2001:44:46 2014/08/08 05:32:08 - Get data
from XML.0 - Server returned HTTP response code: 500 for URL:
<URL>StartDate=2014-07-30%2001:44:46 Nested exception: Server returned
HTTP response code: 500 for URL: <URL>StartDate=2014-07-30%2001:44:46
2014/08/08 05:32:08 - Get data from XML.0 -  2014/08/08 05:32:08 - Get
data from XML.0 -   at
org.pentaho.di.trans.steps.getxmldata.GetXMLData.setDocument(GetXMLData.java:184)
2014/08/08 05:32:08 - Get data from XML.0 -     at
org.pentaho.di.trans.steps.getxmldata.GetXMLData.ReadNextString(GetXMLData.java:409)
2014/08/08 05:32:08 - Get data from XML.0 -     at
org.pentaho.di.trans.steps.getxmldata.GetXMLData.getXMLRowPutRowWithErrorhandling(GetXMLData.java:712)
2014/08/08 05:32:08 - Get data from XML.0 -     at
org.pentaho.di.trans.steps.getxmldata.GetXMLData.getXMLRow(GetXMLData.java:698)
2014/08/08 05:32:08 - Get data from XML.0 -     at
org.pentaho.di.trans.steps.getxmldata.GetXMLData.processRow(GetXMLData.java:655)
2014/08/08 05:32:08 - Get data from XML.0 -     at
org.pentaho.di.trans.step.RunThread.run(RunThread.java:60) 2014/08/08
05:32:08 - Get data from XML.0 -    at
java.lang.Thread.run(Thread.java:722) 2014/08/08 05:32:08 - Get data
from XML.0 - Caused by: org.dom4j.DocumentException: Server returned
HTTP response code: 500 for URL: <URL>StartDate=2014-07-30%2001:44:46
Nested exception: Server returned HTTP response code: 500 for URL:
<URL>StartDate=2014-07-30%2001:44:46 2014/08/08 05:32:08 - Get data
from XML.0 -    at org.dom4j.io.SAXReader.read(SAXReader.java:484)
2014/08/08 05:32:08 - Get data from XML.0 -     at
org.dom4j.io.SAXReader.read(SAXReader.java:291) 2014/08/08 05:32:08 -
Get data from XML.0 -   at
org.pentaho.di.trans.steps.getxmldata.GetXMLData.setDocument(GetXMLData.java:162)
2014/08/08 05:32:08 - Get data from XML.0 -     ... 6 more 2014/08/08
05:32:08 - Get data from XML.0 - Finished processing (I=0, O=0, R=1,
W=0, U=0, E=1) 2014/08/08 05:32:08 - Purchase Sat ETL - ERROR (version
5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) : Errors detected! 2014/08/08 05:32:08 - Purchase Sat ETL - Purchase Sat ETL
2014/08/08 05:32:08 - Purchase Sat ETL - ERROR (version 5.0.1-stable,
build 1 from 2013-11-15_16-08-58 by buildguy) : Errors detected!
2014/08/08 05:32:08 - Purchase Sat ETL - Purchase Sat ETL

从API加载数据时出现问题 - URL - Kettle Pentaho

0 个答案: