我们正在使用新发布的Java/Kotlin API在Dialogflow上实现Google上的操作。
它称为Speech Bank
。
在智能手机上通过帐户关联流程测试时,用户会收到MalformedResponse错误,无法完成流程,因此无法成功切换回常规流程。
日志(以下详细信息)包含MalformedResponse: Failed to parse Dialogflow response into AppResponse because of empty speech response
消息,并且用户在其设备上收到Speech bank isn't responding right now. Try again soon.
消息。
有关我们的设置的更多详细信息:
该操作已配置为使用我们自己的OAuth 2兼容模拟基础结构进行帐户链接。
在Dialogflow中配置了一个意图(称为RawText
),其余的交互作用将由自己的内部应用程序通过其Web挂钩处理。
到目前为止,这是使用Java对状态机进行编码的方式:
public class AoGApp extends DialogflowApp {
private final static Logger log = LoggerFactory.getLogger(AoGApp.class);
public static final String GREETING = "GOOGLE_ASSISTANT_WELCOME";
@ForIntent("RawText")
//@ForIntent("actions.intent.MAIN")
public ActionResponse launchRequestHandler(ActionRequest request) {
String userId = request.getAppRequest().getUser().getUserId();
log.info("userId={}",userId);
String queryText = request.getWebhookRequest().getQueryResult().getQueryText();
log.info("queryText={}", queryText);
String speech = null;
ResponseBuilder responseBuilder = getResponseBuilder(request);
if (isBlank(userId) || GREETING.equalsIgnoreCase(queryText)) {
speech = "\nHi. I sense a great banking experience in your future, I see that your account isn't connected. "
+ "I've sent a link to your Google Assistant app that will get you started and set up in just several simple steps. "
+ "Don't worry, I'll be here waiting, just summon me when you're ready.";
responseBuilder.add(
new SignIn()
.setContext(speech));
} else {
speech = "Welcome. You can say hello.";
responseBuilder.add(speech);
}
return responseBuilder.build();
}
@ForIntent("actions.intent.SIGN_IN")
public ActionResponse getSignInStatus(ActionRequest request) {
ResponseBuilder responseBuilder = getResponseBuilder(request);
String text = "Hello from sign-in handler";
responseBuilder.add(text);
log.info(text);
return responseBuilder.build();
}
}
and the associated HttpRequest processing:
@Override
protected void handlePOST(final Request request, final HttpServletResponse response) {
try {
String rawRequest = ControllerUtils.toString(request.getReader());
String jsonResponse = app.handleRequest(rawRequest, getHeadersMap(request)).get();
log.info("Generated response:\n {}", ControllerUtils.prettyPrint(jsonResponse));
response.setContentType(APPLICATION_JSON.getMimeType());
response.getWriter().write(jsonResponse);
} catch (Exception e) {
handleError(response, e);
}
}
public final class ControllerUtils {
private final static Logger log = LoggerFactory.getLogger(ControllerUtils.class);
private static ObjectMapper mapper = new ObjectMapper();
public static String toString(BufferedReader reader) throws Exception {
String rawRequest = reader
.lines()
//.map(e -> e.concat(System.lineSeparator()))
.collect(Collectors.joining(System.lineSeparator()));
log.info("Received AoG Request {}",rawRequest);
return rawRequest;
}
public static String prettyPrint(String json) throws Exception {
return mapper.writerWithDefaultPrettyPrinter().writeValueAsString(mapper.readValue(json, Object.class));
}
public static Map<String, String> getHeadersMap(org.eclipse.jetty.server.Request jettyRequest){
return Collections.list((Enumeration<String>) jettyRequest.getHeaderNames())
.stream()
.collect(Collectors.toMap(
name -> name,
jettyRequest::getHeader));
}
}
按照上面的配置,OAuth授权代码流将执行正常的OAuth 2步骤:
点击/login
端点以提供凭据
命中/token
端点以获取令牌(以下日志中其值为token1
。我们具有生成和注入自己的令牌的功能,这是一个测试环境,因此我们产生了这个token1
值,该值似乎已成功合并到后续请求中。)
下面是失败的交互的详细屏幕快照,以及由Google控制台上的操作提供的附件日志:
[
{
"textPayload": "Sending request with post data: {\"user\":{\"userId\":\"ABwppHFQHUBr0RrWA_OuL-kK2sxTPUvQtL3D-x2Ydr-7uxLt9zzEFzJrGB-X96d9XY8k9XTJj-RUg9WpzGB9jg\",\"locale\":\"en-US\",\"lastSeen\":\"2019-02-20T21:32:22Z\",\"userStorage\":\"{\\\"data\\\":{}}\"},\"conversation\":{\"conversationId\":\"ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug\",\"type\":\"NEW\"},\"inputs\":[{\"intent\":\"actions.intent.MAIN\",\"rawInputs\":[{\"inputType\":\"VOICE\",\"query\":\"open speech Bank\"}]}],\"surface\":{\"capabilities\":[{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.MEDIA_RESPONSE_AUDIO\"},{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.WEB_BROWSER\"}]},\"isInSandbox\":true,\"availableSurfaces\":[{\"capabilities\":[{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.WEB_BROWSER\"}]}]}.",
"insertId": "f9fzrtf3hjgn4",
"resource": {
"type": "assistant_action",
"labels": {
"project_id": "speechbank-e8a15",
"version_id": "",
"action_id": "actions.intent.MAIN"
}
},
"timestamp": "2019-02-21T13:47:56.713587946Z",
"severity": "DEBUG",
"labels": {
"channel": "preview",
"source": "AOG_REQUEST_RESPONSE",
"querystream": "GOOGLE_USER"
},
"logName": "projects/speechbank-e8a15/logs/actions.googleapis.com%2Factions",
"trace": "projects/366800784520/traces/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug",
"receiveTimestamp": "2019-02-21T13:47:57.205496026Z"
},
{
"textPayload": "Received response from agent with body: HTTP/1.1 200 OK\r\nServer: nginx/1.13.6\r\nDate: Thu, 21 Feb 2019 13:47:57 GMT\r\nContent-Type: application/json;charset=UTF-8\r\nContent-Length: 426\r\nX-Cloud-Trace-Context: d8cb97627afa1d2977b9f567f29598de/11157405402824233090;o=0\r\nGoogle-Actions-API-Version: 2\r\nX-SHARD: shard-2\r\nVia: 1.1 google\r\nAlt-Svc: clear\r\n\r\n{\"conversationToken\":\"[\\\"_actions_on_google\\\"]\",\"expectUserResponse\":true,\"expectedInputs\":[{\"inputPrompt\":{},\"possibleIntents\":[{\"intent\":\"actions.intent.SIGN_IN\",\"inputValueData\":{\"@type\":\"type.googleapis.com/google.actions.v2.SignInValueSpec\"}}]}],\"responseMetadata\":{\"status\":{\"message\":\"Success (200)\"},\"queryMatchInfo\":{\"queryMatched\":true,\"intent\":\"f645f492-f6dc-4e7e-8da6-45711c654ad0\"}},\"userStorage\":\"{\\\"data\\\":{}}\"}.",
"insertId": "f9fzrtf3hjgn5",
"resource": {
"type": "assistant_action",
"labels": {
"version_id": "",
"action_id": "actions.intent.MAIN",
"project_id": "speechbank-e8a15"
}
},
"timestamp": "2019-02-21T13:47:57.190979036Z",
"severity": "DEBUG",
"labels": {
"source": "AOG_REQUEST_RESPONSE",
"querystream": "GOOGLE_USER",
"channel": "preview"
},
"logName": "projects/speechbank-e8a15/logs/actions.googleapis.com%2Factions",
"trace": "projects/366800784520/traces/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug",
"receiveTimestamp": "2019-02-21T13:47:57.205496026Z"
},
{
"textPayload": "Sending request with post data: {\"user\":{\"userId\":\"ABwppHFQHUBr0RrWA_OuL-kK2sxTPUvQtL3D-x2Ydr-7uxLt9zzEFzJrGB-X96d9XY8k9XTJj-RUg9WpzGB9jg\",\"accessToken\":\"token1\",\"locale\":\"en-US\",\"lastSeen\":\"2019-02-20T21:32:22Z\",\"userStorage\":\"{\\\"data\\\":{}}\"},\"conversation\":{\"conversationId\":\"ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug\",\"type\":\"ACTIVE\",\"conversationToken\":\"[\\\"_actions_on_google\\\"]\"},\"inputs\":[{\"intent\":\"actions.intent.SIGN_IN\",\"rawInputs\":[{}],\"arguments\":[{\"name\":\"SIGN_IN\",\"extension\":{\"@type\":\"type.googleapis.com/google.actions.v2.SignInValue\",\"status\":\"OK\"}},{\"name\":\"text\"}]}],\"surface\":{\"capabilities\":[{\"name\":\"actions.capability.WEB_BROWSER\"},{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.MEDIA_RESPONSE_AUDIO\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"}]},\"isInSandbox\":true,\"availableSurfaces\":[{\"capabilities\":[{\"name\":\"actions.capability.WEB_BROWSER\"},{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"}]}]}.",
"insertId": "120k9w1f3jmw55",
"resource": {
"type": "assistant_action",
"labels": {
"version_id": "",
"action_id": "actions.intent.SIGN_IN",
"project_id": "speechbank-e8a15"
}
},
"timestamp": "2019-02-21T13:48:28.768213970Z",
"severity": "DEBUG",
"labels": {
"source": "AOG_REQUEST_RESPONSE",
"querystream": "GOOGLE_USER",
"channel": "preview"
},
"logName": "projects/speechbank-e8a15/logs/actions.googleapis.com%2Factions",
"trace": "projects/366800784520/traces/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug",
"receiveTimestamp": "2019-02-21T13:48:28.912828815Z"
},
{
"textPayload": "Received response from agent with body: HTTP/1.1 200 OK\r\nServer: nginx/1.13.6\r\nDate: Thu, 21 Feb 2019 13:48:28 GMT\r\nContent-Type: application/json;charset=UTF-8\r\nContent-Length: 570\r\nX-Cloud-Trace-Context: 664d8fdaf9cd3d880d41f11ac2176e0e/16724608154084655134;o=0\r\nGoogle-Actions-API-Version: 2\r\nAssistant-Interaction-Error-Code: -1\r\nAssistant-Interaction-Error-Message: Failed to parse Dialogflow response into AppResponse because of empty speech response\r\nX-SHARD: shard-2\r\nVia: 1.1 google\r\nAlt-Svc: clear\r\n\r\n{\n \"responseMetadata\": {\n \"status\": {\n \"code\": 10,\n \"message\": \"Failed to parse Dialogflow response into AppResponse because of empty speech response\",\n \"details\": [{\n \"@type\": \"type.googleapis.com/google.protobuf.Value\",\n \"value\": \"{\\\"id\\\":\\\"5d4bed8d-c58c-4429-9838-f758d6f335f2\\\",\\\"timestamp\\\":\\\"2019-02-21T13:48:28.806Z\\\",\\\"lang\\\":\\\"en-us\\\",\\\"result\\\":{},\\\"status\\\":{\\\"code\\\":200,\\\"errorType\\\":\\\"success\\\"},\\\"sessionId\\\":\\\"ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug\\\"}\"\n }]\n }\n }\n}.",
"insertId": "120k9w1f3jmw56",
"resource": {
"type": "assistant_action",
"labels": {
"project_id": "speechbank-e8a15",
"version_id": "",
"action_id": "actions.intent.SIGN_IN"
}
},
"timestamp": "2019-02-21T13:48:28.899033790Z",
"severity": "DEBUG",
"labels": {
"channel": "preview",
"source": "AOG_REQUEST_RESPONSE",
"querystream": "GOOGLE_USER"
},
"logName": "projects/speechbank-e8a15/logs/actions.googleapis.com%2Factions",
"trace": "projects/366800784520/traces/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug",
"receiveTimestamp": "2019-02-21T13:48:28.912828815Z"
},
{
"textPayload": "MalformedResponse: Failed to parse Dialogflow response into AppResponse because of empty speech response",
"insertId": "1b6j2e6f39jvuy",
"resource": {
"type": "assistant_action",
"labels": {
"project_id": "speechbank-e8a15",
"version_id": "",
"action_id": "actions.intent.SIGN_IN"
}
},
"timestamp": "2019-02-21T13:48:28.899403302Z",
"severity": "ERROR",
"labels": {
"channel": "preview",
"source": "JSON_RESPONSE_VALIDATION",
"querystream": "GOOGLE_USER"
},
"logName": "projects/speechbank-e8a15/logs/actions.googleapis.com%2Factions",
"trace": "projects/366800784520/traces/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug",
"receiveTimestamp": "2019-02-21T13:48:28.914061262Z"
}
]
根据上述设置说明,任何人都可以帮助我们找出导致 MalformedResponse 异常的原因以及需要进行哪些更改以消除它。
这种异常情况太晦涩,以至于引发了无数问题,使人们不得不从哪里着手处理。我在这里只列出一些,非常感谢您提供一些指导。
AoG中的意图名称和Dialogflow之间是否存在任何关联?他们应该遵循任何命名约定吗?错误的原因可能在于他们被错误命名了吗?
是否可以将MalformedResponse解释为响应中缺少特定字段?由于Google选择公开不同消息格式(Dialogflow和AppResponse)之间转换的内部工作,因此Dialogflow响应中是否需要列出某些字段?
这是否意味着即使在这种情况下正在传递的OAuth消息也需要包含某些语音?
最初,从Dialogflow接收到的userId
似乎总是null
,但是查询文本似乎填充了GOOGLE_ASSISTANT_WELCOME
,因此我们开始启动帐户链接流程逻辑基于null
的假设。这是正确的假设吗?
在哪种情况下userId
将首先被填充(例如在Alexa中,在为用户启用技能后会自动生成),以便上述else
条件可以被触发?
由身份验证基础结构发布并由AoG支持的OAuth令牌应采用任何特定格式,即OIDC或JWT。可以是任何随机字符串吗? token1
仍然是AoG说法中的有效令牌(就像Alexa一样)吗?
是否配置了错误的Java意图处理程序?我们应该对AoG帐户关联流程的响应中的哪个意图名称做出反应?
是否有一个包含所有含义的名称,可以将其处理程序合并到Java应用程序中以促进对上述内容的进一步调试?
“空语音响应”是什么意思,我们没有提供哪些期望的值并导致中断?
我们已配置的所有配置都不应配置?
如果有什么关系,请参阅以下来自Webhook的日志:
[java] 02-21-2019 13:47:57 [qtp2056234595-127] INFO domain.lola.user.utils.http.ControllerUtils [toString:30] - Received AoG Request {
[java] "responseId": "0156911c-d7e8-405b-bf8f-f23320c02030",
[java] "queryResult": {
[java] "queryText": "GOOGLE_ASSISTANT_WELCOME",
[java] "parameters": {
[java] "any": ""
[java] },
[java] "allRequiredParamsPresent": true,
[java] "fulfillmentMessages": [{
[java] "text": {
[java] "text": [""]
[java] }
[java] }],
[java] "outputContexts": [{
[java] "name": "projects/speechbank-e8a15/agent/sessions/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug/contexts/google_assistant_welcome",
[java] "parameters": {
[java] "any.original": "",
[java] "any": ""
[java] }
[java] }, {
[java] "name": "projects/speechbank-e8a15/agent/sessions/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug/contexts/actions_capability_screen_output",
[java] "parameters": {
[java] "any.original": "",
[java] "any": ""
[java] }
[java] }, {
[java] "name": "projects/speechbank-e8a15/agent/sessions/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug/contexts/actions_capability_audio_output",
[java] "parameters": {
[java] "any.original": "",
[java] "any": ""
[java] }
[java] }, {
[java] "name": "projects/speechbank-e8a15/agent/sessions/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug/contexts/google_assistant_input_type_voice",
[java] "parameters": {
[java] "any.original": "",
[java] "any": ""
[java] }
[java] }, {
[java] "name": "projects/speechbank-e8a15/agent/sessions/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug/contexts/actions_capability_web_browser",
[java] "parameters": {
[java] "any.original": "",
[java] "any": ""
[java] }
[java] }, {
[java] "name": "projects/speechbank-e8a15/agent/sessions/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug/contexts/actions_capability_media_response_audio",
[java] "parameters": {
[java] "any.original": "",
[java] "any": ""
[java] }
[java] }],
[java] "intent": {
[java] "name": "projects/speechbank-e8a15/agent/intents/f645f492-f6dc-4e7e-8da6-45711c654ad0",
[java] "displayName": "RawText"
[java] },
[java] "intentDetectionConfidence": 1.0,
[java] "languageCode": "en-us"
[java] },
[java] "originalDetectIntentRequest": {
[java] "source": "google",
[java] "version": "2",
[java] "payload": {
[java] "isInSandbox": true,
[java] "surface": {
[java] "capabilities": [{
[java] "name": "actions.capability.AUDIO_OUTPUT"
[java] }, {
[java] "name": "actions.capability.MEDIA_RESPONSE_AUDIO"
[java] }, {
[java] "name": "actions.capability.SCREEN_OUTPUT"
[java] }, {
[java] "name": "actions.capability.WEB_BROWSER"
[java] }]
[java] },
[java] "inputs": [{
[java] "rawInputs": [{
[java] "query": "open speech Bank",
[java] "inputType": "VOICE"
[java] }],
[java] "intent": "actions.intent.MAIN"
[java] }],
[java] "user": {
[java] "userStorage": "{\"data\":{}}",
[java] "lastSeen": "2019-02-20T21:32:22Z",
[java] "locale": "en-US",
[java] "userId": "ABwppHFQHUBr0RrWA_OuL-kK2sxTPUvQtL3D-x2Ydr-7uxLt9zzEFzJrGB-X96d9XY8k9XTJj-RUg9WpzGB9jg"
[java] },
[java] "conversation": {
[java] "conversationId": "ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug",
[java] "type": "NEW"
[java] },
[java] "availableSurfaces": [{
[java] "capabilities": [{
[java] "name": "actions.capability.AUDIO_OUTPUT"
[java] }, {
[java] "name": "actions.capability.SCREEN_OUTPUT"
[java] }, {
[java] "name": "actions.capability.WEB_BROWSER"
[java] }]
[java] }]
[java] }
[java] },
[java] "session": "projects/speechbank-e8a15/agent/sessions/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug"
[java] }
[java] 02-21-2019 13:47:57 [qtp2056234595-127] INFO domain.lola.user.utils.actionsongoogle.AoGApp [launchRequestHandler:26] - userId=null
[java] 02-21-2019 13:47:57 [qtp2056234595-127] INFO domain.lola.user.utils.actionsongoogle.AoGApp [launchRequestHandler:28] - queryText=GOOGLE_ASSISTANT_WELCOME
[java] 02-21-2019 13:47:57 [qtp2056234595-127] INFO domain.lola.user.utils.actionsongoogle.AoGBotService [handlePOST:103] - Generated response:
[java] {
[java] "outputContexts" : [ {
[java] "lifespanCount" : 99,
[java] "name" : "projects/speechbank-e8a15/agent/sessions/ABwppHE35s8T6qSdaaCMNiWuMdY7UsvQ3sHbLZJOQkVA4AFD2nhKuqTTvoJTkVh7yU81GCHbvlfZTxmULLd4Ug/contexts/_actions_on_google",
[java] "parameters" : {
[java] "data" : "{}"
[java] }
[java] } ],
[java] "payload" : {
[java] "google" : {
[java] "expectUserResponse" : true,
[java] "isSsml" : false,
[java] "systemIntent" : {
[java] "intent" : "actions.intent.SIGN_IN",
[java] "data" : {
[java] "@type" : "type.googleapis.com/google.actions.v2.SignInValueSpec"
[java] }
[java] },
[java] "userStorage" : "{\"data\":{}}"
[java] }
[java] }
[java] }
答案 0 :(得分:1)
此行
responseBuilder.add(new SignIn().setContext(speech));
将使用SIGN_IN事件创建您的回复。 因此,在您的dialogFlow中,您需要使用 actions_intent_SIGN_IN 添加另一个意图, 并且在Java中还需要实现它,here您可以找到更多信息。