我想从网络流式传输音频，然后使用python google-cloud-speech API将其转换为文本。我已经将其集成到我的Django频道代码中。
对于前端，我直接复制了此code，后端具有此代码（请参见下文）。现在，问题出了，我没有收到任何异常或错误，但是我没有从Google API得到任何结果...
我尝试过的事情：
process函数的for循环内，该控件从未到达该循环内。 我已经遍历了Java代码here，并试图了解......我已经在本地设置了Java代码并对其进行了调试。我了解的一件事是在Java代码中，方法onWebSocketBinary正在接收一个整数数组，从前端我们像这样发送
socket.send(Int16Array.from(floatSamples.map(function (n) {return n * MAX_INT;})));
在Java中，它们正在转换为字节串，然后发送给Google。在django中，我放了调试点，并注意到我正在以二进制字符串形式获取数据...所以，我觉得我不需要做任何事情...但是，我尝试了几种方法将其转换为整数数组，但这没用，因为google本身期望以字节为单位...（您可以在下面看到带注释的代码）
有人可以帮我吗？
Django代码：
import json
from channels.generic.websocket import WebsocketConsumer
# Imports the Google Cloud client library
from google.cloud import speech
from google.cloud.speech import enums
from google.cloud.speech import types
# Instantiates a client
client = speech.SpeechClient()
language_code = "en-US"
streaming_config = None
class SpeechToTextConsumer(WebsocketConsumer):
def connect(self):
self.accept()
def disconnect(self, close_code):
pass
def process(self, streaming_recognize_response: types.StreamingRecognitionResult):
for response in streaming_recognize_response:
if not response.results:
continue
result = response.results[0]
self.send(text_data=json.dumps(result))
def receive(self, text_data=None, bytes_data=None):
global streaming_config
if text_data:
data = json.loads(text_data)
rate = data["sampleRate"]
config = types.RecognitionConfig(
encoding=enums.RecognitionConfig.AudioEncoding.LINEAR16,
sample_rate_hertz=rate,
language_code=language_code,
)
streaming_config = types.StreamingRecognitionConfig(
config=config, interim_results=True, single_utterance=False
)
types.StreamingRecognizeRequest(streaming_config=streaming_config)
self.send(text_data=json.dumps({"message": "processing..."}))
if bytes_data:
# bytes_data = bytes_data[math.floor(len(bytes_data) / 2) :]
# bytes_data = bytes_data.lstrip(b"\x00")
# bytes_data = int.from_bytes(bytes_data, "little")
stream = [bytes_data]
requests = (
types.StreamingRecognizeRequest(audio_content=chunk) for chunk in stream
)
responses = client.streaming_recognize(streaming_config, requests)
self.process(responses)
答案 0 :(得分：1)
我在创建虚拟人工智能助手时遇到了类似的问题，我相信我至少可以提供一些帮助。我绝不是专家，但我确实找到了一种方法来实现 Google 的文本转语音引擎。我使用了 python 的语音识别库（您可以使用 pip install speech_recognition 下载）并将其导入为“sr”。从这里开始，您可以使用recognize.recognize_google(audio file) 设置Google 的API。您不需要帐户，因为该库已经包含一个密钥，并且在任何地方（例如 Django）都非常容易设置和实施。这是一个非常有用的教程链接，我真的很recommend。这是 documentation 的链接。这是一个有用的 program，它获取音频文件并使用所有可用的语音识别服务转录它。这是下面的代码，你可以使用你喜欢的任何服务，sphinx离线运行，而且google的API不需要注册，因为它已经有了密钥和密码。
#!/usr/bin/env python3
import speech_recognition as sr
# obtain path to "english.wav" in the same folder as this script
from os import path
AUDIO_FILE = path.join(path.dirname(path.realpath(__file__)), "english.wav")
# AUDIO_FILE = path.join(path.dirname(path.realpath(__file__)), "french.aiff")
# AUDIO_FILE = path.join(path.dirname(path.realpath(__file__)), "chinese.flac")
# use the audio file as the audio source
r = sr.Recognizer()
with sr.AudioFile(AUDIO_FILE) as source:
audio = r.record(source) # read the entire audio file
# recognize speech using Sphinx
try:
print("Sphinx thinks you said " + r.recognize_sphinx(audio))
except sr.UnknownValueError:
print("Sphinx could not understand audio")
except sr.RequestError as e:
print("Sphinx error; {0}".format(e))
# recognize speech using Google Speech Recognition
try:
# for testing purposes, we're just using the default API key
# to use another API key, use `r.recognize_google(audio, key="GOOGLE_SPEECH_RECOGNITION_API_KEY")`
# instead of `r.recognize_google(audio)`
print("Google Speech Recognition thinks you said " + r.recognize_google(audio))
except sr.UnknownValueError:
print("Google Speech Recognition could not understand audio")
except sr.RequestError as e:
print("Could not request results from Google Speech Recognition service; {0}".format(e))
# recognize speech using Google Cloud Speech
GOOGLE_CLOUD_SPEECH_CREDENTIALS = r"""INSERT THE CONTENTS OF THE GOOGLE CLOUD SPEECH JSON CREDENTIALS FILE HERE"""
try:
print("Google Cloud Speech thinks you said " + r.recognize_google_cloud(audio, credentials_json=GOOGLE_CLOUD_SPEECH_CREDENTIALS))
except sr.UnknownValueError:
print("Google Cloud Speech could not understand audio")
except sr.RequestError as e:
print("Could not request results from Google Cloud Speech service; {0}".format(e))
# recognize speech using Wit.ai
WIT_AI_KEY = "INSERT WIT.AI API KEY HERE" # Wit.ai keys are 32-character uppercase alphanumeric strings
try:
print("Wit.ai thinks you said " + r.recognize_wit(audio, key=WIT_AI_KEY))
except sr.UnknownValueError:
print("Wit.ai could not understand audio")
except sr.RequestError as e:
print("Could not request results from Wit.ai service; {0}".format(e))
# recognize speech using Microsoft Azure Speech
AZURE_SPEECH_KEY = "INSERT AZURE SPEECH API KEY HERE" # Microsoft Speech API keys 32-character lowercase hexadecimal strings
try:
print("Microsoft Azure Speech thinks you said " + r.recognize_azure(audio, key=AZURE_SPEECH_KEY))
except sr.UnknownValueError:
print("Microsoft Azure Speech could not understand audio")
except sr.RequestError as e:
print("Could not request results from Microsoft Azure Speech service; {0}".format(e))
# recognize speech using Microsoft Bing Voice Recognition
BING_KEY = "INSERT BING API KEY HERE" # Microsoft Bing Voice Recognition API keys 32-character lowercase hexadecimal strings
try:
print("Microsoft Bing Voice Recognition thinks you said " + r.recognize_bing(audio, key=BING_KEY))
except sr.UnknownValueError:
print("Microsoft Bing Voice Recognition could not understand audio")
except sr.RequestError as e:
print("Could not request results from Microsoft Bing Voice Recognition service; {0}".format(e))
# recognize speech using Houndify
HOUNDIFY_CLIENT_ID = "INSERT HOUNDIFY CLIENT ID HERE" # Houndify client IDs are Base64-encoded strings
HOUNDIFY_CLIENT_KEY = "INSERT HOUNDIFY CLIENT KEY HERE" # Houndify client keys are Base64-encoded strings
try:
print("Houndify thinks you said " + r.recognize_houndify(audio, client_id=HOUNDIFY_CLIENT_ID, client_key=HOUNDIFY_CLIENT_KEY))
except sr.UnknownValueError:
print("Houndify could not understand audio")
except sr.RequestError as e:
print("Could not request results from Houndify service; {0}".format(e))
# recognize speech using IBM Speech to Text
IBM_USERNAME = "INSERT IBM SPEECH TO TEXT USERNAME HERE" # IBM Speech to Text usernames are strings of the form XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
IBM_PASSWORD = "INSERT IBM SPEECH TO TEXT PASSWORD HERE" # IBM Speech to Text passwords are mixed-case alphanumeric strings
try:
print("IBM Speech to Text thinks you said " + r.recognize_ibm(audio, username=IBM_USERNAME, password=IBM_PASSWORD))
except sr.UnknownValueError:
print("IBM Speech to Text could not understand audio")
except sr.RequestError as e:
print("Could not request results from IBM Speech to Text service; {0}".format(e))
希望这在某种程度上有所帮助！