我正在尝试使用bing speech api将调用功能添加到我的机器人中，使用语音识别到我的机器人。 当我运行我的应用程序时，它在运行时给我一个例外（在标题中给出）。我不知道如何处理它以及导致它的原因。 它发生在SendToBot方法的以下行。
await postToBot.PostAsync(activity, CancellationToken.None);
Microsoft.Bot.Builder.Internals.Fibers.NullWait`1 [[Microsoft.Bot.Builder.Dialogs.Internals.DialogTask， Microsoft.Bot.Builder，Version = 3.5.5.0，Culture = neutral， 公钥= 31bf3856ad364e35]]＆＃39;在Assembly＆＃39; Microsoft.Bot.Builder中， Version = 3.5.5.0，Culture = neutral，PublicKeyToken = 31bf3856ad364e35＆＃39;是 没有标记为可序列化。
堆栈追踪：
at System.Runtime.Serialization.FormatterServices.InternalGetSerializableMembers(RuntimeType type)
at System.Runtime.Serialization.FormatterServices.GetSerializableMembers(Type type, StreamingContext context)
at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitMemberInfo()
at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitSerialize(Object obj, ISurrogateSelector surrogateSelector, StreamingContext context, SerObjectInfoInit serObjectInfoInit, IFormatterConverter converter, ObjectWriter objectWriter, SerializationBinder binder)
at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo)
at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, Header[] inHeaders, __BinaryWriter serWriter, Boolean fCheck)
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph, Header[] headers, Boolean fCheck)
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph)
at Microsoft.Bot.Builder.Internals.Fibers.FormatterStore`1.Microsoft.Bot.Builder.Internals.Fibers.IStore<T>.Save(T item)
at Microsoft.Bot.Builder.Internals.Fibers.ErrorResilientStore`1.Microsoft.Bot.Builder.Internals.Fibers.IStore<T>.Save(T item)
at Microsoft.Bot.Builder.Internals.Fibers.FactoryStore`1.Microsoft.Bot.Builder.Internals.Fibers.IStore<T>.Save(T item)
at Microsoft.Bot.Builder.Dialogs.Internals.DialogTask.<Microsoft-Bot-Builder-Base-IEventLoop-PollAsync>d__23.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Bot.Builder.Dialogs.Internals.ReactiveDialogTask.<Microsoft-Bot-Builder-Base-IEventLoop-PollAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Bot.Builder.Dialogs.Internals.ScoringEventLoop`1.<Microsoft-Bot-Builder-Base-IEventLoop-PollAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Bot.Builder.Dialogs.Internals.EventLoopDialogTask.<Microsoft-Bot-Builder-Dialogs-Internals-IPostToBot-PostAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Bot.Builder.Dialogs.Internals.SetAmbientThreadCulture.<Microsoft-Bot-Builder-Dialogs-Internals-IPostToBot-PostAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Bot.Builder.Dialogs.Internals.PersistentDialogTask.<Microsoft-Bot-Builder-Dialogs-Internals-IPostToBot-PostAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.Bot.Builder.Dialogs.Internals.PersistentDialogTask.<Microsoft-Bot-Builder-Dialogs-Internals-IPostToBot-PostAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Bot.Builder.Dialogs.Internals.ExceptionTranslationDialogTask.<Microsoft-Bot-Builder-Dialogs-Internals-IPostToBot-PostAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Bot.Builder.Dialogs.Internals.SerializeByConversation.<Microsoft-Bot-Builder-Dialogs-Internals-IPostToBot-PostAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Bot.Builder.Dialogs.Internals.PostUnhandledExceptionToUser.<Microsoft-Bot-Builder-Dialogs-Internals-IPostToBot-PostAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.Bot.Builder.Dialogs.Internals.PostUnhandledExceptionToUser.<Microsoft-Bot-Builder-Dialogs-Internals-IPostToBot-PostAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Bot.Builder.Dialogs.Internals.LogPostToBot.<Microsoft-Bot-Builder-Dialogs-Internals-IPostToBot-PostAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at JoeBot.BingSpeech.<SendToBot>d__14.MoveNext() in D:\PARIS\JoeBot\CallingBot\BingSpeech.cs:line 123
在BingSpeech类中的以下方法（最后给出）
private async Task SendToBot(RecognizedPhrase recognizedPhrase)
{
Activity activity = new Activity()
{
From = new ChannelAccount { Id = conversationResult.Id },
Conversation = new ConversationAccount { Id = conversationResult.Id },
Recipient = new ChannelAccount { Id = "Bot" },
ServiceUrl = "https://skype.botframework.com",
ChannelId = "skype",
};
activity.Text = recognizedPhrase.DisplayText;
using (var scope = Microsoft.Bot.Builder.Dialogs.Conversation
.Container.BeginLifetimeScope(DialogModule.LifetimeScopeTag, Configure))
{
scope.Resolve<IMessageActivity>
(TypedParameter.From((IMessageActivity)activity));
DialogModule_MakeRoot.Register
(scope, () => new LUISDialogClass());
var postToBot = scope.Resolve<IPostToBot>();
await postToBot.PostAsync(activity, CancellationToken.None);
}
}
我的LUISDialogClass（）被标记为Serializable。 LUIS是根对话框，它根据意图/实体匹配将请求转发到进一步的WikipediaDialog类。我也在这里使用静态类和字段创建JSON字符串（出于特殊目的/将其发送到服务器）。
[Serializable]
public class LUISDialogClass : LuisDialog<object>
{
[LuisIntent("")]
[LuisIntent("None")]
public async Task None(IDialogContext context, LuisResult result)
{
var message = result;
await context.PostAsync("Intent match : None");
context.Wait(MessageReceived);
}
[LuisIntent("FindInfo")]
public async Task FindInfo(IDialogContext context, LuisResult result)
{
try
{
var message = result;
for (int i = 1; i < 2; i++)
{
JSONDataObjects.data[i] = new Datum();
JSONDataObjects.data[i].dataKind = "luisResponse";
JSONDataObjects.data[i].query = message.Query;
JSONDataObjects.data[i].intents = new Intent[message.Intents.Count];
JSONDataObjects.data[i].entities = new Entity[message.Entities.Count];
for (int j = 0; j < message.Intents.Count; j++)
{
JSONDataObjects.data[i].intents[j] = new Intent();
JSONDataObjects.data[i].intents[j].intent = message.Intents[j]?.Intent;
JSONDataObjects.data[i].intents[j].score = (float)message.Intents[j]?.Score;
}
for (int j = 0; j < message.Entities.Count; j++)
{
JSONDataObjects.data[i].entities[j] = new Entity();
JSONDataObjects.data[i].entities[j].entity = message.Entities[j]?.Entity;
JSONDataObjects.data[i].entities[j].type = message.Entities[j]?.Type;
JSONDataObjects.data[i].entities[j].startIndex = (int)message.Entities[j]?.StartIndex;
JSONDataObjects.data[i].entities[j].endIndex = (int)message.Entities[j]?.EndIndex;
JSONDataObjects.data[i].entities[j].score = (float)message.Entities[j]?.Score;
}
}
// See if the intent has a > .70 match
bool boolIntentMatch = false;
foreach (var objIntent in message.Intents)
{
// If the FindInfo Intent is detected
// and it's score is greater than or = to .70
// set boolIntentMatch = true
if (
(objIntent.Intent == "FindInfo")
&& (objIntent.Score >= .70f)
)
{
boolIntentMatch = true;
}
}
if (boolIntentMatch)
{
// ** To Do: Code to handle a Match **
string item = "";
EntityRecommendation rec;
if (message.TryFindEntity("objects", out rec))
{
item = rec.Entity;
}
else if (message.TryFindEntity("location", out rec))
{
item = rec.Entity;
}
Activity messageActivity = (Activity)context.MakeMessage();
messageActivity.Text = item;
messageActivity.ChannelId = "emulator";
await context.Forward(new WikipediaDialog(), ResumeAfterWikipedia, messageActivity, System.Threading.CancellationToken.None);
}
else
{
// Not a match -- Ask to rephrase the question
await context.PostAsync("Please try to rephrase your question. Not a good intent match found");
}
}
catch (Exception ex)
{
//Debug.WriteLine(ex);
await context.PostAsync($"Failed and caught in FindInfo LUIS Dialog, with Exception: {ex.Message}");
}
}
private async Task ResumeAfterWikipedia(IDialogContext context, IAwaitable<object> result)
{
try
{
var message = await result;
}
catch (Exception ex)
{
await context.PostAsync($"Failed and caught in ResumeAfterWikipedia Dialog, with Exception: {ex.Message}");
}
finally
{
context.Wait(this.MessageReceived);
}
}
}
}
我正在使用CallingController类来处理调用，如下所示
[BotAuthentication]
// Prefix route for your calling controller
[RoutePrefix("api/calling")]
public class CallingController : ApiController
{
public CallingController()
: base()
{
CallingConversation.RegisterCallingBot(c => new JoeCallingBot(c));
}
// Callback route for Skype calling events.
// Make sure to set the Microsoft.Bot.Builder.Calling.CallbackUrl in web.config
[Route("callback")]
public async Task<HttpResponseMessage> ProcessCallingEventAsync()
{
return await CallingConversation.SendAsync(Request, CallRequestType.CallingEvent);
}
// Route for incoming call events
[Route("call")]
public async Task<HttpResponseMessage> ProcessIncomingCallAsync()
{
return await CallingConversation.SendAsync(Request, CallRequestType.IncomingCall);
}
}
我的JoeCallingBot类正在关注
public class JoeCallingBot : ICallingBot
{
public ICallingBotService CallingBotService
{
get; private set;
}
private List<string> response = new List<string>();
int silenceTimes = 0;
bool sttFailed = false;
public JoeCallingBot(ICallingBotService callingBotService)
{
if (callingBotService == null)
throw new ArgumentNullException(nameof(callingBotService));
this.CallingBotService = callingBotService;
CallingBotService.OnIncomingCallReceived += OnIncomingCallReceived;
CallingBotService.OnPlayPromptCompleted += OnPlayPromptCompleted;
CallingBotService.OnRecordCompleted += OnRecordCompleted;
CallingBotService.OnHangupCompleted += OnHangupCompleted;
}
private Task OnIncomingCallReceived(IncomingCallEvent incomingCallEvent)
{
var id = Guid.NewGuid().ToString();
incomingCallEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
new Answer { OperationId = id },
GetRecordForText("Welcome! How can I help you?")
};
return Task.FromResult(true);
}
private ActionBase GetRecordForText(string promptText)
{
PlayPrompt prompt;
if (string.IsNullOrEmpty(promptText))
prompt = null;
else
prompt = GetPromptForText(promptText);
var id = Guid.NewGuid().ToString();
return new Record()
{
OperationId = id,
PlayPrompt = prompt,
MaxDurationInSeconds = 10,
InitialSilenceTimeoutInSeconds = 5,
MaxSilenceTimeoutInSeconds = 2,
PlayBeep = false,
RecordingFormat = RecordingFormat.Wav,
StopTones = new List<char> { '#' }
};
}
private Task OnPlayPromptCompleted(PlayPromptOutcomeEvent playPromptOutcomeEvent)
{
if (response.Count > 0)
{
silenceTimes = 0;
var actionList = new List<ActionBase>();
foreach (var res in response)
{
Debug.WriteLine($"Response ----- {res}");
}
actionList.Add(GetPromptForText(response));
actionList.Add(GetRecordForText(string.Empty));
playPromptOutcomeEvent.ResultingWorkflow.Actions = actionList;
response.Clear();
}
else
{
if (sttFailed)
{
playPromptOutcomeEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
GetRecordForText("I didn't catch that, would you kindly repeat?")
};
sttFailed = false;
silenceTimes = 0;
}
else if (silenceTimes > 2)
{
playPromptOutcomeEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
GetPromptForText("Something went wrong. Call again later."),
new Hangup() { OperationId = Guid.NewGuid().ToString() }
};
playPromptOutcomeEvent.ResultingWorkflow.Links = null;
silenceTimes = 0;
}
else
{
silenceTimes++;
playPromptOutcomeEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
GetSilencePrompt(2000)
};
}
}
return Task.CompletedTask;
}
private async Task OnRecordCompleted(RecordOutcomeEvent recordOutcomeEvent)
{
if (recordOutcomeEvent.RecordOutcome.Outcome == Outcome.Success)
{
var record = await recordOutcomeEvent.RecordedContent;
BingSpeech bs = new BingSpeech(recordOutcomeEvent.ConversationResult, t => response.Add(t), s => sttFailed = s);
bs.CreateDataRecoClient();
bs.SendAudioHelper(record);
recordOutcomeEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
GetSilencePrompt()
};
}
else
{
if (silenceTimes > 1)
{
recordOutcomeEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
GetPromptForText("Thank you for calling"),
new Hangup() { OperationId = Guid.NewGuid().ToString() }
};
recordOutcomeEvent.ResultingWorkflow.Links = null;
silenceTimes = 0;
}
else
{
silenceTimes++;
recordOutcomeEvent.ResultingWorkflow.Actions = new List<ActionBase>
{
GetRecordForText("I didn't catch that, would you kinly repeat?")
};
}
}
}
private Task OnHangupCompleted(HangupOutcomeEvent hangupOutcomeEvent)
{
hangupOutcomeEvent.ResultingWorkflow = null;
return Task.FromResult(true);
}
private static PlayPrompt GetPromptForText(string text)
{
var prompt = new Prompt { Value = text, Voice = VoiceGender.Female };
return new PlayPrompt { OperationId = Guid.NewGuid().ToString(), Prompts = new List<Prompt> { prompt } };
}
private static PlayPrompt GetPromptForText(List<string> text)
{
var prompts = new List<Prompt>();
foreach (var txt in text)
{
if (!string.IsNullOrEmpty(txt))
prompts.Add(new Prompt { Value = txt, Voice = VoiceGender.Female });
}
if (prompts.Count == 0)
return GetSilencePrompt(1000);
return new PlayPrompt { OperationId = Guid.NewGuid().ToString(), Prompts = prompts };
}
private static PlayPrompt GetSilencePrompt(uint silenceLengthInMilliseconds = 3000)
{
var prompt = new Prompt { Value = string.Empty, Voice = VoiceGender.Female, SilenceLengthInMilliseconds = silenceLengthInMilliseconds };
return new PlayPrompt { OperationId = Guid.NewGuid().ToString(), Prompts = new List<Prompt> { prompt } };
}
}
我的完整BingSpeech课程如下
public class BingSpeech
{
private DataRecognitionClient dataClient;
private Action<string> _callback;
private ConversationResult conversationResult;
private Action<bool> _failedCallback;
public BingSpeech(ConversationResult conversationResult, Action<string> callback, Action<bool> failedCallback)
{
this.conversationResult = conversationResult;
_callback = callback;
_failedCallback = failedCallback;
}
public string DefaultLocale { get; } = "en-US";
public string SubscriptionKey { get; } = "Bing Speech API KEY"; //Bing Speech Recognition Key
public void CreateDataRecoClient()
{
this.dataClient = SpeechRecognitionServiceFactory.CreateDataClient(
SpeechRecognitionMode.ShortPhrase,
this.DefaultLocale,
this.SubscriptionKey);
this.dataClient.OnResponseReceived += this.OnDataShortPhraseResponseReceivedHandler;
}
public void SendAudioHelper(Stream recordedStream)
{
// Note for wave files, we can just send data from the file right to the server.
// In the case you are not an audio file in wave format, and instead you have just
// raw data (for example audio coming over bluetooth), then before sending up any
// audio data, you must first send up an SpeechAudioFormat descriptor to describe
// the layout and format of your raw audio data via DataRecognitionClient's sendAudioFormat() method.
int bytesRead = 0;
byte[] buffer = new byte[1024];
try
{
do
{
// Get more Audio data to send into byte buffer.
bytesRead = recordedStream.Read(buffer, 0, buffer.Length);
// Send of audio data to service.
this.dataClient.SendAudio(buffer, bytesRead);
}
while (bytesRead > 0);
}
catch (Exception ex)
{
WriteLine("Exception ------------ " + ex.Message);
}
finally
{
// We are done sending audio. Final recognition results will arrive in OnResponseReceived event call.
this.dataClient.EndAudio();
}
}
private async void OnDataShortPhraseResponseReceivedHandler(object sender, SpeechResponseEventArgs e)
{
this.WriteLine("--- OnDataShortPhraseResponseReceivedHandler ---");
this.WriteResponseResult(e);
// we got the final result, so it we can end the mic reco. No need to do this
// for dataReco, since we already called endAudio() on it as soon as we were done
// sending all the data.
// Send to bot
if (e.PhraseResponse.RecognitionStatus == RecognitionStatus.RecognitionSuccess)
{
await SendToBot(e.PhraseResponse.Results
.OrderBy(k => k.Confidence)
.FirstOrDefault());
}
else
{
_failedCallback(true);
}
}
private async Task SendToBot(RecognizedPhrase recognizedPhrase)
{
Activity activity = new Activity()
{
From = new ChannelAccount { Id = conversationResult.Id },
Conversation = new ConversationAccount { Id = conversationResult.Id },
Recipient = new ChannelAccount { Id = "Bot" },
ServiceUrl = "https://skype.botframework.com",
ChannelId = "skype",
};
activity.Text = recognizedPhrase.DisplayText;
using (var scope = Microsoft.Bot.Builder.Dialogs.Conversation
.Container.BeginLifetimeScope(DialogModule.LifetimeScopeTag, Configure))
{
scope.Resolve<IMessageActivity>
(TypedParameter.From((IMessageActivity)activity));
DialogModule_MakeRoot.Register
(scope, () => new LUISDialogClass());
var postToBot = scope.Resolve<IPostToBot>();
try
{
await postToBot.PostAsync(activity, CancellationToken.None);
}
catch (SerializationException ex)
{
Debug.WriteLine("Exception: " + ex.StackTrace);
}
}
}
private void Configure(ContainerBuilder builder)
{
builder.Register(c => new BotToUserSpeech(c.Resolve<IMessageActivity>(), _callback))
.As<IBotToUser>()
.InstancePerLifetimeScope();
}
private void WriteResponseResult(SpeechResponseEventArgs e)
{
if (e.PhraseResponse.Results.Length == 0)
{
this.WriteLine("No phrase response is available.");
}
else
{
this.WriteLine("********* Final n-BEST Results *********");
for (int i = 0; i < e.PhraseResponse.Results.Length; i++)
{
this.WriteLine(
"[{0}] Confidence={1}, Text=\"{2}\"",
i,
e.PhraseResponse.Results[i].Confidence,
e.PhraseResponse.Results[i].DisplayText);
}
this.WriteLine(string.Empty);
}
}
private void WriteLine(string format, params object[] args)
{
var formattedStr = string.Format(format, args);
Trace.WriteLine(formattedStr);
Debug.WriteLine(formattedStr);
}
}
流程如下： 调用来自CallingController类，它将它转发给JoeCallingBot类，后者将它转发给BingSpeech类，后来转发它的LUISDialog类
使用Bot.Builder版本3.3.3解决了这个问题。但是较新版本的Bot Builder SDK导致了这个问题。 Github上有一个未解决的问题：
答案 0 :(得分：0)
将Bot Builder升级到3.9.x解决了问题：）