Question

我正在尝试制作类似的东西，我正在录制声音，并且基于声音（音高，频率，不确定），图像应该移动。

我能够实现录制，也可以实现图像序列，但是可以单独进行。我不知道如何链接，只是为了信息，我试图实现类似的东西嘴巴移动应用程序： app url here

我的问题是，如何根据声音频率移动/动画图像。

由于

Answer 1

我完成了解决方案。使用Dirac并解决了问题。

编辑：

这是什么？

DiracAudioPlayer是一组新的Cocoa类，它以一种方便的方式包装整个Dirac功能，暴露出类似于AVAudioPlayer提供的API。请注意，这不是AVAudioPlayer子类。以下是API的核心功能和说明。

DiracAudioPlayer核心功能

DiracAudioPlayer是一组类，允许基于文件的各种音频格式（包括MPMediaItems）播放，同时实时改变音频文件的速度和音高。版本3.6由DiracAudioPlayerBase（负责文件IO和回放的基类），DiracAudioPlayer（包装Dirac Core API）和DiracFxAudioPlayer（包装DiracFx API）组成。

确保在项目中包含所有3个类以及“ExtAudioFile”和“util”文件夹，并将Accelerate.framework和CoreAudio.framework添加到项目中。在MacOS X上你也必须添加AudioUnit.framework，在iOS上你必须添加AudioToolbox.framework，AVFoundation.framework，MediaPlayer.framework和CoreMedia.framework。

DiracAudioPlayer是......

...与苹果兼容的类可播放适用于iOS（版本4及更高版本）和MacOS X（版本10.6及更高版本）的时间拉伸音频 ......非常好用 ......完全兼容ARC ...交付给您，包括完整的源代码

DiracAudioPlayer API

2012年11月发布的3.6版提供以下呼叫：

- (id) initWithContentsOfURL:(NSURL*)inUrl channels:(int)channels error: (NSError **)error;

Initializes and returns an audio player for playing a designated sound file. A URL identifying the sound file to play. The audio data must be in a format supported by Core Audio. Pass in the address of a nil-initialized NSError object. If an error occurs, upon return the NSError object describes the error. To use an item from the user's iPod library supply the URL that you get via MPMediaItem's MPMediaItemPropertyAssetURL property as inUrl. Note that FairPlay protected content can NOT be processed.


- (void) setDelegate:(id)delegate;
- (id) delegate;

设置/获取类的委托。如果您实现委托协议，DiracAudioPlayer将调用您的

实现

- (void)diracPlayerDidFinishPlaying:(DiracAudioPlayerBase *)player successfully:(BOOL)flag

播放完毕后

- (void) changeDuration:(float)duration;
- (void) changePitch:(float)pitch;

更改播放速度和音高

- (NSInteger) numberOfLoops;
- (void) setNumberOfLoops:(NSInteger)loops;

A value of 0, which is the default, means to play the sound once. Set a positive integer value to specify the number of times to return to the start and play again. For example, specifying a value of 1 results in a total of two plays of the sound. Set any negative integer value to loop the sound indefinitely until you call the stop method.

- (void) updateMeters;

必须在调用-peakPowerForChannel之前调用，以便更新其内部测量值

- (float) peakPowerForChannel:(NSUInteger)channelNumber;

A floating-point representation, in decibels, of a given audio channel’s current peak power. A return value of 0 dB indicates full scale, or maximum power; a return value of -160 dB indicates minimum power (that is, near silence). If the signal provided to the audio player exceeds ±full scale, then the return value may exceed 0 (that is, it may enter the positive range). To obtain a current peak power value, you must call the updateMeters method before calling this method.


- (BOOL) prepareToPlay;

Starts the Dirac processing thread and prepares the sound file for playback. If you don't call this explicitly it will be called when calling -play


- (NSUInteger) numberOfChannels;

与音频播放器关联的声音中的音频通道数。（只读）

- (NSTimeInterval) fileDuration;

返回与音频播放器关联的声音的总持续时间（以秒为单位）。（只读）

- (NSTimeInterval) currentTime;
- (void) setCurrentTime:(NSTimeInterval)time

Returns the current play time in the input file. Note that if you apply time stretching, -currentTime will reflect the slowed down time depending on the time stretch factor. 

IMPORTANT CHANGE: In previous versions this value returned the total play time independent of the position in the file. Please update your code accordingly to reflect the change

Setting this property causes playback to fast forward or rewind to the specified play time.


- (void) play;

Plays a sound asynchronously. Returns YES on success, or NO on failure. Calling this method implicitly calls the -prepareToPlay method if the audio player is not already prepared to play.

- (NSURL*) url;

与音频播放器关联的声音的URL。（只读）

- (void) setVolume:(float)volume;
- (float) volume;

音频播放器的播放增益，范围从0.0到1.0。

- (BOOL) playing;

A Boolean value that indicates whether the audio player is playing (YES) or not (NO). (read-only). To find out when playback has stopped, use the diracPlayerDidFinishPlaying:successfully: delegate method.

- (void) pause;

Pauses playback; sound remains ready to resume playback from where it left off. Calling pause leaves the audio player prepared to play; it does not release the audio hardware that was acquired upon calling -play or -prepareToPlay.

- (void) stop;

Stops playback and undoes the setup needed for playback. Calling this method, or allowing a sound to finish playing, undoes the setup performed upon calling the -play or -prepareToPlay methods.

Answer 2

大多数文字转语音系统都允许您注册一个回调函数，该函数会向您发送正在生成的音素（通常是声音）。请查看以下link。点击左侧的回调。向下看SpeechPhonemeProcPtr，这将允许您注册一个函数，当发出的噪音为“呃”，“th”，“啊”或任何噪音时，将调用该函数。然后，您可以更新图像，使其看起来就像人们在制作特定声音时的嘴巴样子。这在IBM的ViaVoice中非常简单，我从未在iPhone上编写过这样的应用程序，但我认为这比尝试匹配音频要好。

如果这是真正未经过滤的音频，那么您可以将其传递给语音识别系统并将识别的文本传递到TTS系统并获取音素。

根据声音频率移动图像？

2 个答案: