但是得到如下错误

Question

我们正在集群模式下构建wso2am。是否有关于构建wso2am-analytics集群的文档？我试过用wso2das，参考如下。 https://docs.wso2.com/display/DAS310/Working+with+Product+Specific+Analytics+Profiles

但是得到如下错误

TID: [-1234] [] [2016-12-09 15:00:00,101] ERROR {org.wso2.carbon.analytics.spark.core.AnalyticsTask} -  Error while executing the scheduled task for the script: APIM_LATENCY_BREAKDOWN_STATS {org.wso2.carbon.analytics.spark.core.AnalyticsTask}
org.wso2.carbon.analytics.spark.core.exception.AnalyticsExecutionException: Exception in executing query CREATE TEMPORARY TABLE APIMGT_PERHOUR_EXECUTION_TIME USING CarbonAnalytics OPTIONS(tableName "ORG_WSO2_APIMGT_STATISTICS_PERHOUREXECUTIONTIMES", schema "    year INT -i, month INT -i, day INT -i, hour INT -i, context STRING, api_version STRING, api STRING, tenantDomain STRING,    apiPublisher STRING, apiResponseTime DOUBLE, securityLatency DOUBLE, throttlingLatency DOUBLE, requestMediationLatency DOUBLE, responseMediationLatency DOUBLE,    backendLatency DOUBLE, otherLatency DOUBLE, firstEventTime LONG, _timestamp LONG -i",    primaryKeys "year, month, day, hour, context, api_version, tenantDomain, apiPublisher",    incrementalProcessing "APIMGT_PERHOUR_EXECUTION_TIME, DAY",    mergeSchema "false")
        at org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQueryLocal(SparkAnalyticsExecutor.java:764)
        at org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQuery(SparkAnalyticsExecutor.java:721)
        at org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeQuery(CarbonAnalyticsProcessorService.java:201)
        at org.wso2.carbon.analytics.spark.core.CarbonAnalyticsProcessorService.executeScript(CarbonAnalyticsProcessorService.java:151)
        at org.wso2.carbon.analytics.spark.core.AnalyticsTask.execute(AnalyticsTask.java:60)
        at org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:67)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Unknown options : incrementalprocessing
        at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider.checkParameters(AnalyticsRelationProvider.java:123)
        at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider.setParameters(AnalyticsRelationProvider.java:113)
        at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider.createRelation(AnalyticsRelationProvider.java:75)
        at org.wso2.carbon.analytics.spark.core.sources.AnalyticsRelationProvider.createRelation(AnalyticsRelationProvider.java:45)
        at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:158)
        at org.apache.spark.sql.execution.datasources.CreateTempTableUsing.run(ddl.scala:92)
        at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:58)
        at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:56)
        at org.apache.spark.sql.execution.ExecutedCommand.doExecute(commands.scala:70)
        at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:132)
        at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$5.apply(SparkPlan.scala:130)
        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
        at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:130)
        at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:55)
        at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:55)
        at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:145)
        at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:130)
        at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:52)
        at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:817)
        at org.wso2.carbon.analytics.spark.core.internal.SparkAnalyticsExecutor.executeQueryLocal(SparkAnalyticsExecutor.java:760)
        ... 11 more

=============================================== =======

任何建议都将受到赞赏!!

Answer 1

只是要清楚你在问什么。您使用的是DAS还是wso2am-2.0.0附带的wso2am-analytics？

如果您使用wso2am-2.0.0以及wso2das-3.1.0，那么DAS附带的spark分析脚本就会出现问题。

这是由于incremental processing的使用。 IncrementalProcessing应更改为IncrementalParams。

您可以看到这已由wso2 here修复，但尚未发布。

您可以在主/批处理分析/脚本

下的DAS碳控制台更新脚本

Answer 2

请回复！我们需要在statdb中保留汇总数据，那么应该清除哪个表？我在http://www.rukspot.com/Publishing_APIM_1100_Runtime_Statistics_to_DAS.html下面找到了一个链接它提到只需清除表格

ORG_WSO2_APIMGT_STATISTICS_DESTINATION

ORG_WSO2_APIMGT_STATISTICS_FAULT

ORG_WSO2_APIMGT_STATISTICS_REQUEST

ORG_WSO2_APIMGT_STATISTICS_RESPONSE

ORG_WSO2_APIMGT_STATISTICS_WORKFLOW

ORG_WSO2_APIMGT_STATISTICS_THROTTLE

如何集群wso2am-analytics-2.0.0

但是得到如下错误

2 个答案: