使用pocketsphinx识别器进行语音反馈

时间:2015-03-17 20:54:32

标签: java android eclipse text-to-speech pocketsphinx-android

仍然使用相同的项目,这是Run pocketSphinx and Google TTS together的延续。我已根据 Nikolay Shymyrev 的指南进行了修订,并做了很多帮助。但我想要实现的最后一个功能仍然存在。谷歌TTS现在运行得很好,但识别器有一些问题。

如果谷歌TTS有些话很长,那么识别器就不会开始。

Speaker.speak("Drive mode now will be enabled. I will read your new messages for you now.");

然后我的onPartialResult如果条件不能像

那样实现
if (text.equals("exit")) {
        speaker.speak("yeah get in");
        recognizer.cancel();
        Intent i = new Intent(getApplicationContext(),PocketSphinxActivity.class);
        startActivity(i);

我认为识别器总是会听,因为它在后台运行,然后它会听取谷歌TTS句子,导致它后来无法识别我的演讲。因为当我使用免提麦克风时,speaker.Speak的句子只是"Drive mode enabled",它会很好地识别我的单词,并在我说"exit"时执行上面的if条件。但是,当句子很长,如"Drive mode now will be enabled, I will read bla bla bla"时,它就不会听我的"exit"字。

我现在要做的是为识别器添加超时以超时几次,以便它不会识别任何不必要的声音。我想把

startRecognition("search", timeout)

但我的Eclipse不会让我这样做。它给了我错误。我使用PocketSphinx for Android 5 pre alpha。

又来了,我的代码构建只是为了测试并确保它只识别"退出"字

SMSReaderMain.java

public class SMSReaderMain extends Activity implements RecognitionListener {    

    private final int CHECK_CODE = 0x1; 
    private final int LONG_DURATION = 5000;
    private final int SHORT_DURATION = 1200;

    private Speaker speaker;


    private TextView smsText;
    private TextView smsSender; 
    private BroadcastReceiver smsReceiver;


    public static final String TURNON_SR = "drive mode";
    public static final String TURNOFF_SR = "ok";
    public static final String DESTROY_SR = "exit";


    public SpeechRecognizer recognizer;
    public HashMap<String, Integer> captions;




    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        setContentView(R.layout.main_sms);  
        captions = new HashMap<String, Integer>();


        new AsyncTask<Void, Void, Exception>() {
            @Override
            protected Exception doInBackground(Void... params) {
                try {
                    Assets assets = new Assets(SMSReaderMain.this);
                    File assetDir = assets.syncAssets();
                    setupRecognizer(assetDir);
                } catch (IOException e) {
                    return e;
                }
                return null;
            }   
            @Override
            protected void onPostExecute(Exception result) {
                if (result != null) {
                    ((TextView) findViewById(R.id.caption_text))
                            .setText("Failed to init recognizer " + result);
                } else {
                    switchSearch(TURNOFF_SR);
                }
            }
        }.execute();

        //toggle = (ToggleButton)findViewById(R.id.speechToggle);
        smsText = (TextView)findViewById(R.id.sms_text);
        smsSender = (TextView)findViewById(R.id.sms_sender);

        checkTTS();
    }

    private void startDriveMode(){
        speaker.allow(true);
        //speaker.speak(getString(R.string.start_speaking));
        speaker.speak("Drive mode enabled");


    }

    private void checkTTS(){
        Intent check = new Intent();
        check.setAction(TextToSpeech.Engine.ACTION_CHECK_TTS_DATA);
        startActivityForResult(check, CHECK_CODE);
    }

    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if(requestCode == CHECK_CODE){
            if(resultCode == TextToSpeech.Engine.CHECK_VOICE_DATA_PASS)
            {
                speaker = new Speaker(this);
            }else {
                Intent install = new Intent();
                install.setAction(TextToSpeech.Engine.ACTION_INSTALL_TTS_DATA);
                startActivity(install);
            }
        }

        startDriveMode();
        initializeSMSReceiver();
        registerSMSReceiver();

    }


    private void initializeSMSReceiver(){
        smsReceiver = new BroadcastReceiver(){
            @Override
            public void onReceive(Context context, Intent intent) {

                Bundle bundle = intent.getExtras();
                if(bundle!=null){
                    Object[] pdus = (Object[])bundle.get("pdus");
                    for(int i=0;i<pdus.length;i++){
                        byte[] pdu = (byte[])pdus[i];
                        SmsMessage message = SmsMessage.createFromPdu(pdu);
                        String text = message.getDisplayMessageBody();
                        String sender = getContactName(message.getOriginatingAddress());
                        speaker.pause(LONG_DURATION);
                        speaker.speak("You have a new message from" + sender + "!");
                        speaker.pause(SHORT_DURATION);
                        speaker.speak(text);
                        smsSender.setText("Message from " + sender);
                        smsText.setText(text);
                    }
                }

            }           
        };      
    }

    private void registerSMSReceiver() {    
        IntentFilter intentFilter = new IntentFilter("android.provider.Telephony.SMS_RECEIVED");
        registerReceiver(smsReceiver, intentFilter);
    }

    private String getContactName(String phone){
        Uri uri = Uri.withAppendedPath(PhoneLookup.CONTENT_FILTER_URI, Uri.encode(phone));
        String projection[] = new String[]{ContactsContract.Data.DISPLAY_NAME};
        Cursor cursor = getContentResolver().query(uri, projection, null, null, null);              
        if(cursor.moveToFirst()){
            return cursor.getString(0);
        }else {
            return "unknown number";
        }
    }

    @Override
    protected void onDestroy() {    
        super.onDestroy();
        unregisterReceiver(smsReceiver);
        speaker.destroy();
    }

    public void onPartialResult(Hypothesis hypothesis) {
        //System.out.println("masuk coiii");
        String text = hypothesis.getHypstr();
        try {
        if (text.equals("exit")) {
            speaker.speak("yeah get in");
            recognizer.cancel();
            Intent i = new Intent(getApplicationContext(),PocketSphinxActivity.class);
            startActivity(i);

        }


        //Intent i= null;
        /**if (text.equals(TURNON_SR)) {
            recognizer.cancel();
            popPicture();
            startDriveMode();   

        }
        if (text.equals(TURNOFF_SR)) {
            //speaker = new Speaker(this);
            speaker.speak(getString(R.string.stop_speaking));
            speaker.allow(false);   

            //popPicture2();
        }

        if (text.equals(DESTROY_SR)) {
            recognizer.cancel();
            i = new Intent(getApplicationContext(),PocketSphinxActivity.class);
            startActivity(i);
            onDestroy();
            //popPicture2();
        } **/

    } catch (Exception e) {
        e.printStackTrace();
    }
 }
    public void popPicture() {
        LayoutInflater inflater = getLayoutInflater();
        View layout = inflater.inflate(R.layout.toast_image,(ViewGroup) 
                findViewById(R.id.toast_layout_id));

        Toast toast = new Toast(getApplicationContext());
        toast.setGravity(Gravity.CENTER_HORIZONTAL, 0, 0);
        toast.setDuration(Toast.LENGTH_SHORT);
        toast.setView(layout);
        toast.show();
    }

    public void onResult(Hypothesis hypothesis) {
        ((TextView) findViewById(R.id.result_text)).setText("");
        if (hypothesis != null) {
            String text = hypothesis.getHypstr();
            makeText(getApplicationContext(), text, Toast.LENGTH_SHORT).show();
        }
    }

    public void switchSearch(String searchName) {
        recognizer.stop();
        recognizer.startListening(searchName);
        //taro timout disini biar mic ga denger suara hp sendiri

        ((TextView) findViewById(R.id.caption_text)).setText(searchName);
    }

    public void setupRecognizer(File assetsDir) {

        File modelsDir = new File(assetsDir, "models");
        recognizer = defaultSetup()
                .setAcousticModel(new File(modelsDir, "hmm/en-us-semi"))
                .setDictionary(new File(modelsDir, "dict/cmu07a.dic"))
                .setRawLogDir(assetsDir).setKeywordThreshold(1e-10f)
                .getRecognizer();
        recognizer.addListener(this);



        // Create grammar-based searches.
        // recognizer.addKeyphraseSearch(TURNOFF_SR, TURNON_SR);
        //recognizer.addGrammarSearch(TURNON_SR, new File(modelsDir, "grammar/sms.gram"));
        //recognizer.addGrammarSearch(TURNOFF_SR, new File(modelsDir, "grammar/sms.gram"));
        //recognizer.addGrammarSearch(DESTROY_SR, new File(modelsDir, "grammar/sms.gram"));

        File menuGrammar = new File(modelsDir, "grammar/sms.gram");
        recognizer.addGrammarSearch(TURNOFF_SR, menuGrammar);
        //recognizer.addGrammarSearch(TURNON_SR, menuGrammar);
        //recognizer.addGrammarSearch(DESTROY_SR, menuGrammar);

    }

    @Override
    public void onBeginningOfSpeech() {
        // TODO Auto-generated method stub

    }

    @Override
    public void onEndOfSpeech() {
        // TODO Auto-generated method stub

    }
}

Speaker.java

public class Speaker implements OnInitListener {

    private TextToSpeech tts;

    private boolean ready = false;
    private boolean prematureSpeak = false;
    private String ps;

    private boolean allowed = false;

    public Speaker(Context context){
        tts = new TextToSpeech(context, this);      
    }   

    public boolean isAllowed(){
        return allowed;
    }

    //public void allow(boolean allowed){
    public void allow(boolean allowed){
        this.allowed = allowed;
    }

    @Override
    public void onInit(int status) {
        if(status == TextToSpeech.SUCCESS){
            // Change this to match your
            // locale
            tts.setLanguage(Locale.US);
            ready = true;
            if (prematureSpeak)
            {
                speak(ps);
                prematureSpeak = false;
            }
        }else{
            ready = false;
        }
    }


    public void speak(String text){

        // Speak only if the TTS is ready
        // and the user has allowed speech

        if(ready && allowed) {
            HashMap<String, String> hash = new HashMap<String,String>();
            hash.put(TextToSpeech.Engine.KEY_PARAM_STREAM, 
                    String.valueOf(AudioManager.STREAM_NOTIFICATION));
            tts.speak(text, TextToSpeech.QUEUE_ADD, hash);
        }
        else if(!ready) {
            prematureSpeak = true;
            ps = text;
        }
    }

    public void pause(int duration){
        tts.playSilence(duration, TextToSpeech.QUEUE_ADD, null);
    }

    // Free up resources
    public void destroy(){
        tts.shutdown();
    }

    public boolean isSpeaking()
    {
        return tts.isSpeaking();
    }

}

1 个答案:

答案 0 :(得分:0)

您的代码有几个问题:

1)我告诉你使用关键字定位模式,你还在使用语法模式

2)您需要在开始语音反馈之前取消识别器,而不是先说话然后取消

 if (text.equals("exit")) {
    speaker.speak("yeah get in");
    recognizer.cancel();
    ....

你需要先取消然后发言:

 if (text.equals("exit")) {
    recognizer.cancel();
    speaker.speak("yeah get in");
    ....

3)说话人结束后你需要重新启动识别器,但不需要再次运行活动,详见How to know when TTS is finished?

在onUtteranceEnded中进行了这些更改后,您再次启动识别器:

public void onUtteranceCompleted(String utteranceId) {
   recognizer.startSearch("search name");
}

请勿在onPartialResult中重启识别器,等待TTS完成。