Google即时的自定义命令

时间:2016-07-20 13:26:34

标签: java android google-now google-voice google-voice-search

我正在尝试让Google即时接受自定义命令,并在进行特定查询时向我的应用发送意图。

我使用Tasker和Autovoice成功完成了这项工作,但我想在不使用这些应用的情况下也这样做。

我在文档中找到了这个link。我在哪里可以处理不能完成任务的常见意图。

我也尝试过Google提供的语音交互API,这几乎是一回事,但这没有帮助。

有没有人在没有使用Commander,Autovoice或Tasker等其他应用程序的情况下实现了这一目标?

2 个答案:

答案 0 :(得分:19)

Google Now目前不接受'自定义命令'。您详细说明的应用使用AcccessibilityService'黑客'拦截语音命令,或使用有根设备拦截xposed framework

然后他们要么对他们采取行动,同时杀死 Google即时,或者忽略它们并允许Google像往常一样显示其结果。

出于很多原因,这是个坏主意:

  1. Google会找到一种方法来阻止这种类型的互动,如果它变得普遍,因为他们显然不希望他们的Now服务受到负面影响。
  2. 它使用硬编码常量,与Google用于显示语音命令的视图类相关。当然,每次发布都会有所变化。
  3. 黑客破解!
  4. 免责声明完成!使用风险自负......

    您需要在AccessibilityService注册Manifest

        <service
            android:name="com.something.MyAccessibilityService"
            android:enabled="true"
            android:label="@string/label"
            android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE" >
            <intent-filter>
                <action android:name="android.accessibilityservice.AccessibilityService" />
            </intent-filter>
    
            <meta-data
                android:name="android.accessibilityservice"
                android:resource="@xml/accessibilityconfig" />
        </service>
    

    将配置文件添加到res/xml

    <accessibility-service
        xmlns:android="http://schemas.android.com/apk/res/android"
        android:accessibilityEventTypes="typeWindowContentChanged"
        android:accessibilityFeedbackType="feedbackGeneric"
        android:accessibilityFlags="flagIncludeNotImportantViews"
        android:canRetrieveWindowContent="true"
        android:description="@string/accessibility_description"
        android:notificationTimeout="100"
        android:settingsActivity="SettingsActivity"/>
    

    您可以选择添加:

        android:packageNames="xxxxxx"
    

    或通过添加更多事件类型来扩展功能:

        android:accessibilityEventTypes="typeViewTextSelectionChanged|typeWindowContentChanged|typeNotificationStateChanged"
    

    包含以下AccessibilityService示例类:

    /*
     * Copyright (c) 2016 Ben Randall
     *
     * Licensed under the Apache License, Version 2.0 (the "License");
     * you may not use this file except in compliance with the License.
     * You may obtain a copy of the License at
     *
     *   http://www.apache.org/licenses/LICENSE-2.0
     *
     * Unless required by applicable law or agreed to in writing, software
     * distributed under the License is distributed on an "AS IS" BASIS,
     * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     * See the License for the specific language governing permissions and
     * limitations under the License.
     */
    
    package com.your.package;
    
    import android.accessibilityservice.AccessibilityService;
    import android.support.annotation.NonNull;
    import android.support.annotation.Nullable;
    import android.util.Log;
    import android.view.accessibility.AccessibilityEvent;
    import android.view.accessibility.AccessibilityNodeInfo;
    
    
    /**
     * @author benrandall76 AT gmail DOT com
     */
    
    public class MyAccessibilityService extends AccessibilityService {
    
        private final boolean DEBUG = true;
        private final String CLS_NAME = MyAccessibilityService.class.getSimpleName();
    
        private static final String GOOGLE_VOICE_SEARCH_PACKAGE_NAME = "com.google.android.googlequicksearchbox";
        private static final String GOOGLE_VOICE_SEARCH_INTERIM_FIELD = "com.google.android.apps.gsa.searchplate.widget.StreamingTextView";
        private static final String GOOGLE_VOICE_SEARCH_FINAL_FIELD = "com.google.android.apps.gsa.searchplate.SearchPlate";
    
        private static final long COMMAND_UPDATE_DELAY = 1000L;
    
        private long previousCommandTime;
        private String previousCommand = null;
    
        private final boolean EXTRA_VERBOSE = false;
    
        @Override
        protected void onServiceConnected() {
            super.onServiceConnected();
            if (DEBUG) {
                Log.i(CLS_NAME, "onServiceConnected");
            }
        }
    
        @Override
        public void onAccessibilityEvent(final AccessibilityEvent event) {
            if (DEBUG) {
                Log.i(CLS_NAME, "onAccessibilityEvent");
            }
    
            if (event != null) {
    
                switch (event.getEventType()) {
    
                    case AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED:
                        if (DEBUG) {
                            Log.i(CLS_NAME, "onAccessibilityEvent: checking for google");
                        }
    
                        if (event.getPackageName() != null && event.getPackageName().toString().matches(
                                GOOGLE_VOICE_SEARCH_PACKAGE_NAME)) {
                            if (DEBUG) {
                                Log.i(CLS_NAME, "onAccessibilityEvent: checking for google: true");
                                Log.i(CLS_NAME, "onAccessibilityEvent: event.getPackageName: " + event.getPackageName());
                                Log.i(CLS_NAME, "onAccessibilityEvent: event.getClassName: " + event.getClassName());
                            }
    
                            final AccessibilityNodeInfo source = event.getSource();
    
                            if (source != null && source.getClassName() != null) {
    
                                if (source.getClassName().toString().matches(
                                        GOOGLE_VOICE_SEARCH_INTERIM_FIELD)) {
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: className interim: true");
                                        Log.i(CLS_NAME, "onAccessibilityEvent: source.getClassName: " + source.getClassName());
                                    }
    
                                    if (source.getText() != null) {
    
                                        final String text = source.getText().toString();
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: interim text: " + text);
                                        }
    
                                        if (interimMatch(text)) {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: child: interim match: true");
                                            }
    
                                            if (commandDelaySufficient(event.getEventTime())) {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: true");
                                                }
    
                                                if (!commandPreviousMatches(text)) {
                                                    if (DEBUG) {
                                                        Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: false");
                                                    }
    
                                                    previousCommandTime = event.getEventTime();
                                                    previousCommand = text;
    
                                                    killGoogle();
    
                                                    if (DEBUG) {
                                                        Log.e(CLS_NAME, "onAccessibilityEvent: INTERIM PROCESSING: " + text);
                                                    }
    
                                                } else {
                                                    if (DEBUG) {
                                                        Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: true");
                                                    }
                                                }
                                            } else {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: false");
                                                }
                                            }
                                            break;
                                        } else {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: child: interim match: false");
                                            }
                                        }
                                    } else {
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: interim text: null");
                                        }
                                    }
                                } else if (source.getClassName().toString().matches(
                                        GOOGLE_VOICE_SEARCH_FINAL_FIELD)) {
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: className final: true");
                                        Log.i(CLS_NAME, "onAccessibilityEvent: source.getClassName: " + source.getClassName());
                                    }
    
                                    final int childCount = source.getChildCount();
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: childCount: " + childCount);
                                    }
    
                                    if (childCount > 0) {
                                        for (int i = 0; i < childCount; i++) {
    
                                            final String text = examineChild(source.getChild(i));
    
                                            if (text != null) {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: child text: " + text);
                                                }
    
                                                if (finalMatch(text)) {
                                                    if (DEBUG) {
                                                        Log.i(CLS_NAME, "onAccessibilityEvent: child: final match: true");
                                                    }
    
                                                    if (commandDelaySufficient(event.getEventTime())) {
                                                        if (DEBUG) {
                                                            Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: true");
                                                        }
    
                                                        if (!commandPreviousMatches(text)) {
                                                            if (DEBUG) {
                                                                Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: false");
                                                            }
    
                                                            previousCommandTime = event.getEventTime();
                                                            previousCommand = text;
    
                                                            killGoogle();
    
                                                            if (DEBUG) {
                                                                Log.e(CLS_NAME, "onAccessibilityEvent: FINAL PROCESSING: " + text);
                                                            }
    
                                                        } else {
                                                            if (DEBUG) {
                                                                Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: true");
                                                            }
                                                        }
                                                    } else {
                                                        if (DEBUG) {
                                                            Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: false");
                                                        }
                                                    }
                                                    break;
                                                } else {
                                                    if (DEBUG) {
                                                        Log.i(CLS_NAME, "onAccessibilityEvent: child: final match: false");
                                                    }
                                                }
                                            } else {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: child text: null");
                                                }
                                            }
                                        }
                                    }
                                } else {
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: className: unwanted " + source.getClassName());
                                    }
    
                                    if (EXTRA_VERBOSE) {
    
                                        if (source.getText() != null) {
    
                                            final String text = source.getText().toString();
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: unwanted text: " + text);
                                            }
                                        } else {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: unwanted text: null");
                                            }
                                        }
    
                                        final int childCount = source.getChildCount();
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: unwanted childCount: " + childCount);
                                        }
    
                                        if (childCount > 0) {
    
                                            for (int i = 0; i < childCount; i++) {
    
                                                final String text = examineChild(source.getChild(i));
    
                                                if (text != null) {
                                                    if (DEBUG) {
                                                        Log.i(CLS_NAME, "onAccessibilityEvent: unwanted child text: " + text);
                                                    }
                                                }
                                            }
                                        }
                                    }
                                }
                            } else {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "onAccessibilityEvent: source null");
                                }
                            }
                        } else {
                            if (DEBUG) {
                                Log.i(CLS_NAME, "onAccessibilityEvent: checking for google: false");
                            }
                        }
                        break;
                    default:
                        if (DEBUG) {
                            Log.i(CLS_NAME, "onAccessibilityEvent: not interested in type");
                        }
                        break;
                }
            } else {
                if (DEBUG) {
                    Log.i(CLS_NAME, "onAccessibilityEvent: event null");
                }
            }
        }
    
        /**
         * Check if the previous command was actioned within the {@link #COMMAND_UPDATE_DELAY}
         *
         * @param currentTime the time of the current {@link AccessibilityEvent}
         * @return true if the delay is sufficient to proceed, false otherwise
         */
        private boolean commandDelaySufficient(final long currentTime) {
            if (DEBUG) {
                Log.i(CLS_NAME, "commandDelaySufficient");
            }
    
            final long delay = (currentTime - COMMAND_UPDATE_DELAY);
    
            if (DEBUG) {
                Log.i(CLS_NAME, "commandDelaySufficient: delay: " + delay);
                Log.i(CLS_NAME, "commandDelaySufficient: previousCommandTime: " + previousCommandTime);
            }
    
            return delay > previousCommandTime;
        }
    
        /**
         * Check if the previous command/text matches the current text we are considering processing
         *
         * @param text the current text
         * @return true if the text matches the previous text we processed, false otherwise.
         */
        private boolean commandPreviousMatches(@NonNull final String text) {
            if (DEBUG) {
                Log.i(CLS_NAME, "commandPreviousMatches");
            }
    
            return previousCommand != null && previousCommand.matches(text);
        }
    
        /**
         * Check if the interim text matches a command we want to intercept
         *
         * @param text the intercepted text
         * @return true if the text matches a command false otherwise
         */
        private boolean interimMatch(@NonNull final String text) {
            if (DEBUG) {
                Log.i(CLS_NAME, "interimMatch");
            }
            return text.matches("do interim results work");
        }
    
        /**
         * Check if the final text matches a command we want to intercept
         *
         * @param text the intercepted text
         * @return true if the text matches a command false otherwise
         */
        private boolean finalMatch(@NonNull final String text) {
            if (DEBUG) {
                Log.i(CLS_NAME, "finalMatch");
            }
    
            return text.matches("do final results work");
        }
    
        /**
         * Recursively examine the {@link AccessibilityNodeInfo} object
         *
         * @param parent the {@link AccessibilityNodeInfo} parent object
         * @return the extracted text or null if no text was contained in the child objects
         */
        private String examineChild(@Nullable final AccessibilityNodeInfo parent) {
            if (DEBUG) {
                Log.i(CLS_NAME, "examineChild");
            }
    
            if (parent != null) {
    
                for (int i = 0; i < parent.getChildCount(); i++) {
    
                    final AccessibilityNodeInfo nodeInfo = parent.getChild(i);
    
                    if (nodeInfo != null) {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "examineChild: nodeInfo: getClassName: " + nodeInfo.getClassName());
                        }
    
                        if (nodeInfo.getText() != null) {
                            if (DEBUG) {
                                Log.i(CLS_NAME, "examineChild: have text: returning: " + nodeInfo.getText().toString());
                            }
                            return nodeInfo.getText().toString();
                        } else {
                            if (DEBUG) {
                                Log.i(CLS_NAME, "examineChild: text: null: recurse");
                            }
    
                            final int childCount = nodeInfo.getChildCount();
                            if (DEBUG) {
                                Log.i(CLS_NAME, "examineChild: childCount: " + childCount);
                            }
    
                            if (childCount > 0) {
    
                                final String text = examineChild(nodeInfo);
    
                                if (text != null) {
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "examineChild: have recursive text: returning: " + text);
                                    }
                                    return text;
                                } else {
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "examineChild: recursive text: null");
                                    }
                                }
                            }
                        }
                    } else {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "examineChild: nodeInfo null");
                        }
                    }
                }
            } else {
                if (DEBUG) {
                    Log.i(CLS_NAME, "examineChild: parent null");
                }
            }
    
            return null;
        }
    
        /**
         * Kill or reset Google
         */
        private void killGoogle() {
            if (DEBUG) {
                Log.i(CLS_NAME, "killGoogle");
            }
    
            // TODO - Either kill the Google process or send an empty intent to clear current search process
        }
    
        @Override
        public void onInterrupt() {
            if (DEBUG) {
                Log.i(CLS_NAME, "onInterrupt");
            }
        }
    
        @Override
        public void onDestroy() {
            super.onDestroy();
            if (DEBUG) {
                Log.i(CLS_NAME, "onDestroy");
            }
        }
    }
    

    我使这个类尽可能缩进,因此希望更容易理解。

    它执行以下操作:

    1. 检查事件类型的类型是否正确
    2. 检查包裹是否属于Google的“现在”
    3. 检查硬编码类类型的节点信息
    4. 检查临时语音命令,因为它已加载到视图中
    5. 检查加载到视图中的最终语音命令
    6. 递归检查语音命令的视图
    7. 检查事件之间的时差
    8. 检查语音命令是否与先前检测到的语音命令相同
    9. 测试:

      1. 在Android辅助功能设置
      2. 中启用Service
      3. 您的应用程序可能需要重新启动才能使服务正确注册
      4. 启动Google语音识别并说“执行中间结果工作
      5. 退出Google即时
      6. 开始表彰并说“做最终结果工作
      7. 以上将演示来自两个硬编码视图的提取文本/命令。如果您不重新启动Google即时,该命令仍会被检测为临时命令。

        使用提取的语音命令,您需要执行自己的语言匹配,以确定这是否是您感兴趣的命令。如果是,您需要阻止Google说出或显示结果。这是通过杀死Google即时或向其发送空语音搜索意图来实现的,其中包含应clear/reset task的标记。

        你将处于竞争状态,所以你的语言处理需要非常聪明,或者非常基本......

        希望有所帮助。

        修改

        对于那些要求“杀死”Google即时的用户,你需要拥有杀死进程的权限,或者发送一个空的(“”)搜索意图来清除当前搜索:

        public static final String PACKAGE_NAME_GOOGLE_NOW = "com.google.android.googlequicksearchbox";
        public static final String ACTIVITY_GOOGLE_NOW_SEARCH = ".SearchActivity";
        
        /**
         * Launch Google Now with a specific search term to resolve
         *
         * @param ctx        the application context
         * @param searchTerm the search term to resolve
         * @return true if the search term was handled correctly, false otherwise
         */
        public static boolean googleNow(@NonNull final Context ctx, @NonNull final String searchTerm) {
            if (DEBUG) {
                Log.i(CLS_NAME, "googleNow");
            }
        
            final Intent intent = new Intent(Intent.ACTION_WEB_SEARCH);
            intent.setComponent(new ComponentName(PACKAGE_NAME_GOOGLE_NOW,
                    PACKAGE_NAME_GOOGLE_NOW + ACTIVITY_GOOGLE_NOW_SEARCH));
        
            intent.putExtra(SearchManager.QUERY, searchTerm);
            intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK | Intent.FLAG_ACTIVITY_CLEAR_TOP
                    | Intent.FLAG_ACTIVITY_RESET_TASK_IF_NEEDED);
        
            try {
                ctx.startActivity(intent);
                return true;
            } catch (final ActivityNotFoundException e) {
                if (DEBUG) {
                    Log.e(CLS_NAME, "googleNow: ActivityNotFoundException");
                    e.printStackTrace();
                }
            } catch (final Exception e) {
                if (DEBUG) {
                    Log.e(CLS_NAME, "googleNow: Exception");
                    e.printStackTrace();
                }
            }
        
            return false;
        
        }
        

答案 1 :(得分:2)

不是您想听到的,但当前版本的API不允许自定义语音命令:

来自https://developers.google.com/voice-actions/custom-actions

  

注意:我们不接受自定义语音操作请求。请继续关注语音操作 - Google Developers和+ GoogleDevelopers以获取产品更新。