将麦克风输入转换为键盘输入

Question

我有一个键盘踏板（如在钢琴键盘中），我已插入麦克风插孔。我希望能够将输入翻译成击键（例如，当我踏上踏板时，按下＆＃34; V＆＃34;在我的COMPUTER键盘上）。我知道我可以使用Java的AWT机器人来执行击键，但是我在处理麦克风输入时遇到了麻烦（我没有音频处理经验）。

这是我踩踏板并释放踏板的信号：

对于有经验的用户来说，这似乎是一项非常简单的任务，那里的任何人都知道我该怎么做吗？

到目前为止，我基本上都是从地方复制粘贴代码来获取此代码：

我使用javax.sound API将麦克风输入读取为字节。当我踏上踏板时，我试图检测振幅的跳跃......字节转换为短路，并将值与任意高的数字进行比较。它似乎工作但是在一次踏板踩下之后，机器人只是一直按下键（值> 32000）。

        while (true) {
            // Read the next chunk of data from the TargetDataLine.
            numBytesRead = microphone.read(data, 0, data.length);
            // Save this chunk of data.
            out.write(data, 0, numBytesRead);

            byte[] bytes = out.toByteArray();
            short[] shorts = new short[bytes.length/2];
            // to turn bytes to shorts as either big endian or little endian. 
            ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(shorts);

            for (short s : shorts) {
                int value = Math.abs(s);
                if (value > 32000)
                {
                    robot.keyPress(KeyEvent.VK_V);
                    robot.keyRelease(KeyEvent.VK_V);
                    break;
                } else {
                    //robot.keyRelease(KeyEvent.VK_V);
                }
            }
        }

编辑：

它一直在继续，因为我没有清除ByteArrayOutputStream缓冲区，所以随着时间的推移，我不断地读取相同的字节集和新字节。 out.reset（）为我解决了这个问题。

我现在的问题是关于我从踏板上读取的输入，如果我在短时间内按下并释放踏板，我无法正确解释按下或释放。

红色圆圈表示踩下（并保持）踏板时的状态，踏板松开时显示黑色矩形。

正如你所看到的，当它被压低时，它会下降然后迅速增加，然后再逐渐回到0。当它被释放时，它会向上射击，然后迅速下降，然后再次回到0。

我现在用来区分两者的方法是仅在两帧/间隔之间存在较大差异时才记录按下/释放。根据图表，当它是负值时我按下它，当它是正值时我发布它。

我遇到的问题是，当信号处于上升状态时（或当信号处于释放的下降状态时我踩下踏板）时，我松开踏板时，会赢得“won”两个帧之间的差异足以让我使用这种方法。

这可能有助于可视化问题：

我不知道如何通过强大的方式来检测新闻/发布。

这是我输入输入的固定代码，如果有人有兴趣尝试同样的事情（顺便说一下我使用这个踏板：http://www.amazon.co.uk/Cherub-WTB-004-Keyboard-Sustain-Pedal/dp/B000UDVV6E）

import java.io.ByteArrayOutputStream;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;

import javax.sound.sampled.*;

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;

public class PedalToKeyboard {

    public static void main(String[] args) {

        AudioFormat format = new AudioFormat(8000.0f, 16, 1, true, true);
        Robot robot = null;

        try {
            robot = new Robot();

        } catch (AWTException e) {
                e.printStackTrace();
        }

        try {
            TargetDataLine microphone = AudioSystem.getTargetDataLine(format);          
            System.out.println(microphone);

            microphone.open(format);

            ByteArrayOutputStream out = new ByteArrayOutputStream();
            int numBytesRead;
            byte[] data = new byte[microphone.getBufferSize()/5];

            // Begin audio capture.
            microphone.start();

            boolean keep_going = true;
            boolean keyPressed = false;         
            short previousShort = 0;
            while (keep_going) {

                // Read the next chunk of data from the TargetDataLine.
                numBytesRead = microphone.read(data, 0, data.length);

                // Reset the buffer (get rid of previous shit)
                out.reset();

                // Save this chunk of data.
                out.write(data, 0, numBytesRead);

                byte[] bytes = out.toByteArray();
                short[] shorts = new short[bytes.length/2];

                // to turn bytes to shorts as either big endian or little endian. 
                ByteBuffer.wrap(bytes).order(ByteOrder.BIG_ENDIAN).asShortBuffer().get(shorts);

                for (short s : shorts) {

                    // Check if descending or ascending (pedal press is descending, release is ascending)
                    if (s < 0) { // descending                  
                        // make sure drop is large instantaneous drop
                        if (Math.abs(Math.abs(previousShort) - Math.abs(s)) > 10000 && s < -32700) {
                            if (!keyPressed) {
                                keyPressed = true;
                                //robot.keyPress(KeyEvent.VK_V);
                                System.out.println("Pressed: " + s);
                                break;
                            }
                        }
                    } else if (s > 0) { // ascending
                        // make sure increase is large instantaneous increase
                        if (Math.abs(Math.abs(previousShort) - Math.abs(s)) > 10000 && s > 32700) {
                            if (keyPressed) {
                                keyPressed = false;
                                //robot.keyRelease(KeyEvent.VK_V);
                                System.out.println("Released: " + s + "\n");
                                break;
                            }
                        }
                    }

                    previousShort = s;
                }
            }    

        } catch (LineUnavailableException e) {
            e.printStackTrace();
        }
    }
}

Answer 1

好吧我实际上调整了一些值并修正了一些逻辑（我在比较差异时错误地应用了Math.abs）。

这对我来说效果很好，任何有MIDI踏板的人都可以尝试调整参数供你自己使用。

package pedal2keyboard;

import java.io.ByteArrayOutputStream;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;

import javax.sound.sampled.*;

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;

/***
 * Author: Dois Koh
 * Date: 27th October 2015
 * 
 * Gets your microphone signal and you can go do whatever you want with it.
 * Right now, it takes signals from my Cherub WTB-004 Keyboard Sustain Pedal, plugged into
 * my microphone jack, and converts it into key presses (holds down V when depressed,
 * releases V when released)
 */
public class PedalToKeyboard {

    // Robot for performing keyboard actions (pressing V)
    public static Robot robot = null;

    // Currently 8KHz, 16 bit signal (2 bytes), single channel, signed (+ and -) and BIG ENDIAN format
    public static AudioFormat format = new AudioFormat(8000.0f, 16, 1, true, true);

    public static TargetDataLine microphone = null;
    public static boolean pedalPressed = false;

    public static void main(String[] args) {

        try {
            // Initialize robot for later use
            robot = new Robot();

            // Retrieve the line to from which to read in the audio signal
            microphone = AudioSystem.getTargetDataLine(format);

            // Open the line in the specified format -
            // Currently 8KHz, 16 bit signal (2 bytes), single channel, signed (+ and -) and BIG ENDIAN format      
            microphone.open(new AudioFormat(8000.0f, 16, 1, true, true));

            ByteArrayOutputStream out = new ByteArrayOutputStream();
            byte[] data = new byte[microphone.getBufferSize()/8];

            // Begin audio capture.
            microphone.start();

            int numBytesRead = 0;
            short previousShort = 0;

            // Continue until program is manually terminated
            while (true) {

                // Read the next chunk of data from the TargetDataLine.
                numBytesRead = microphone.read(data, 0, data.length);

                // Reset the buffer (get rid of previous data)
                out.reset();

                // Save this chunk of data.
                out.write(data, 0, numBytesRead);

                byte[] bytes = out.toByteArray();
                short[] shorts = new short[bytes.length/2];

                // to turn bytes to shorts as either big endian or little endian. 
                ByteBuffer.wrap(bytes).order(ByteOrder.BIG_ENDIAN).asShortBuffer().get(shorts);

                // Iterate through retrieved 16 bit data (shorts)
                for (short s : shorts) {

                    // Check if descending or ascending (pedal press is descending, release is ascending)
                    if (s < 0) { // descending                  
                        // make sure drop is large instantaneous drop
                        if (Math.abs(previousShort - s) > 200 && s < -32700) {
                            if (!pedalPressed) {
                                PedalPressedAction();
                                previousShort = s;
                                break;
                            }
                        }
                    } else if (s > 0) { // ascending
                        // make sure increase is large instantaneous increase
                        if (Math.abs(previousShort - s) > 200 && s > 32700) {
                            if (pedalPressed) {
                                PedalReleasedAction();
                                previousShort = s;
                                break;
                            }
                        }
                    }

                    previousShort = s;
                }
            }    

        } catch (LineUnavailableException | AWTException e) {
            e.printStackTrace();
        } finally {
            if (microphone != null)
                microphone.close();
        }
    }

    /***
     * The action to perform when the pedal is depressed
     */
    public static void PedalPressedAction() {
        pedalPressed = true;
        robot.keyPress(KeyEvent.VK_V);
    }

    /***
     * The action to perform when the pedal is released
     */
    public static void PedalReleasedAction(){
        pedalPressed = false;
        robot.keyRelease(KeyEvent.VK_V);        
    }
}

Answer 2

如果您查看附加的图表，您可以看到该值在相当长的一段时间内保持在32000以上 - 可能大约100ms左右。根据您的采样率，这可以转化为相当多的按键。例如，在44.1kHz，这将是4410。您最初只能超过阈值时才需要模拟按键。为此，您需要跟踪某些状态。你可以使用PRESS_THRESHOLD和RELEASE_THRESHOLD的值来玩，但是从32000和31000开始。不设置它们的原因是为了防止错误按键，如果信号在边缘附近有毛刺。

for (short s : shorts) {
    int value = Math.abs(s);
    switch (state)
    {
    case State.pressed:
        if (value < RELEASE_THRESHOLD)
        {
            state = State.released;
        }
        break;
    case State.released:
        if (value > PRESS_THRESHOLD)
        {           
            robot.keyPress(KeyEvent.VK_V);
            robot.keyRelease(KeyEvent.VK_V);
            state = State.pressed;
        }
        break;
    }
}