我正在GoPro录像上运行Affectiva SDK 4.0。我正在Ubuntu 16.04上使用C ++程序。 GoPro视频以60 fps录制。问题是Affectiva仅提供约一半帧（即30 fps）的结果。如果我看一下Affectiva提供的时间戳，则最后一个时间戳与视频时长相匹配，这意味着Affectiva会以某种方式跳过每一帧。
在运行Affectiva之前，我使用以下命令运行ffmpeg，以确保视频的帧频恒定为60 fps：
ffmpeg -i in.MP4 -vf -y -vcodec libx264 -preset medium -r 60 -map_metadata 0:g -strict -2 out.MP4 </dev/null 2>&1
当我使用ffprobe -show_entries frame=pict_type,pkt_pts_time -of csv -select_streams v in.MP4检查演示时间戳时，我将获得原始视频的以下值：
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/media/GoPro_concat/GoPro_concat.MP4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.20.100
Duration: 01:14:46.75, start: 0.000000, bitrate: 15123 kb/s
Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1280x720 [SAR 1:1 DAR 16:9], 14983 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 119.88 tbc (default)
Metadata:
handler_name : GoPro AVC
timecode : 13:17:26:44
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
Metadata:
handler_name : GoPro AAC
Stream #0:2(eng): Data: none (tmcd / 0x64636D74)
Metadata:
handler_name : GoPro AVC
timecode : 13:17:26:44
Unsupported codec with id 0 for input stream 2
frame,0.000000,I
frame,0.016683,P
frame,0.033367,P
frame,0.050050,P
frame,0.066733,P
frame,0.083417,P
frame,0.100100,P
frame,0.116783,P
frame,0.133467,I
frame,0.150150,P
frame,0.166833,P
frame,0.183517,P
frame,0.200200,P
frame,0.216883,P
frame,0.233567,P
frame,0.250250,P
frame,0.266933,I
frame,0.283617,P
frame,0.300300,P
frame,0.316983,P
frame,0.333667,P
frame,0.350350,P
frame,0.367033,P
frame,0.383717,P
frame,0.400400,I
frame,0.417083,P
frame,0.433767,P
frame,0.450450,P
frame,0.467133,P
frame,0.483817,P
frame,0.500500,P
frame,0.517183,P
frame,0.533867,I
frame,0.550550,P
frame,0.567233,P
frame,0.583917,P
frame,0.600600,P
frame,0.617283,P
frame,0.633967,P
frame,0.650650,P
frame,0.667333,I
frame,0.684017,P
frame,0.700700,P
frame,0.717383,P
frame,0.734067,P
frame,0.750750,P
frame,0.767433,P
frame,0.784117,P
frame,0.800800,I
frame,0.817483,P
frame,0.834167,P
frame,0.850850,P
frame,0.867533,P
frame,0.884217,P
frame,0.900900,P
frame,0.917583,P
frame,0.934267,I
frame,0.950950,P
frame,0.967633,P
frame,0.984317,P
frame,1.001000,P
frame,1.017683,P
frame,1.034367,P
frame,1.051050,P
frame,1.067733,I
...
我已将完整输出上传到OneDrive。
如果我在原始视频上运行Affectiva（未经ffmpeg处理），则会遇到丢帧的相同问题。我在affdex::VideoDetector detector(60);
ffmpeg命令或Affectiva是否存在问题？
编辑：我想我已经找到问题所在。似乎Affectiva并没有处理整个视频，而是在处理了一定数量的帧后停止，没有任何错误消息。下面我发布了我正在使用的C ++代码。在onProcessingFinished()方法中，我将在处理完成后将某些内容打印到控制台。但是此消息永远不会被打印出来，因此Affectiva永远不会结束。
我的代码有问题吗？还是应该将视频编码为MP4以外的其他格式？
#include "VideoDetector.h"
#include "FrameDetector.h"
#include <iostream>
#include <fstream>
#include <mutex>
#include <condition_variable>
std::mutex m;
std::condition_variable conditional_variable;
bool processed = false;
class Listener : public affdex::ImageListener {
public:
Listener(std::ofstream * fout) {
this->fout = fout;
}
virtual void onImageCapture(affdex::Frame image){
//std::cout << "called";
}
virtual void onImageResults(std::map<affdex::FaceId, affdex::Face> faces, affdex::Frame image){
//std::cout << faces.size() << " faces detected:" << std::endl;
for(auto& kv : faces){
(*this->fout) << image.getTimestamp() << ",";
(*this->fout) << kv.first << ",";
(*this->fout) << kv.second.emotions.joy << ",";
(*this->fout) << kv.second.emotions.fear << ",";
(*this->fout) << kv.second.emotions.disgust << ",";
(*this->fout) << kv.second.emotions.sadness << ",";
(*this->fout) << kv.second.emotions.anger << ",";
(*this->fout) << kv.second.emotions.surprise << ",";
(*this->fout) << kv.second.emotions.contempt << ",";
(*this->fout) << kv.second.emotions.valence << ",";
(*this->fout) << kv.second.emotions.engagement << ",";
(*this->fout) << kv.second.measurements.orientation.pitch << ",";
(*this->fout) << kv.second.measurements.orientation.yaw << ",";
(*this->fout) << kv.second.measurements.orientation.roll << ",";
(*this->fout) << kv.second.faceQuality.brightness << std::endl;
//std::cout << kv.second.emotions.fear << std::endl;
//std::cout << kv.second.emotions.surprise << std::endl;
//std::cout << (int) kv.second.emojis.dominantEmoji;
}
}
private:
std::ofstream * fout;
};
class ProcessListener : public affdex::ProcessStatusListener{
public:
virtual void onProcessingException (affdex::AffdexException ex){
std::cerr << "[Error] " << ex.getExceptionMessage();
}
virtual void onProcessingFinished (){
{
std::lock_guard<std::mutex> lk(m);
processed = true;
std::cout << "[Affectiva] Video processing finised." << std::endl;
}
conditional_variable.notify_one();
}
};
int main(int argc, char ** argsv)
{
affdex::VideoDetector detector(60, 1, affdex::FaceDetectorMode::SMALL_FACES);
//affdex::VideoDetector detector(60, 1, affdex::FaceDetectorMode::LARGE_FACES);
std::string classifierPath="/home/wrafael/affdex-sdk/data";
detector.setClassifierPath(classifierPath);
detector.setDetectAllEmotions(true);
// Output
std::ofstream fout(argsv[2]);
fout << "timestamp" << ",";
fout << "faceId" << ",";
fout << "joy" << ",";
fout << "fear" << ",";
fout << "disgust" << ",";
fout << "sadness" << ",";
fout << "anger" << ",";
fout << "surprise" << ",";
fout << "contempt" << ",";
fout << "valence" << ",";
fout << "engagement" << ",";
fout << "pitch" << ",";
fout << "yaw" << ",";
fout << "roll" << ",";
fout << "brightness" << std::endl;
Listener l(&fout);
ProcessListener pl;
detector.setImageListener(&l);
detector.setProcessStatusListener(&pl);
detector.start();
detector.process(argsv[1]);
// wait for the worker
{
std::unique_lock<std::mutex> lk(m);
conditional_variable.wait(lk, []{return processed;});
}
fout.flush();
fout.close();
}
编辑2 ：现在，我进一步研究了这个问题，只查看了一个持续时间为19分钟53秒的GoPro文件（GoPro拆分了录音）。当我在原始视频上使用affdex::VideoDetector detector(60, 1, affdex::FaceDetectorMode::SMALL_FACES);运行Affectiva时，将产生以下file。 Affectiva在906年代后停止，没有任何错误消息，也没有打印“ [Affectiva]视频处理已完成”。
当我现在使用ffmpeg -i raw.MP4 -y -vcodec libx264 -preset medium -r 60 -map_metadata 0:g -strict -2 out.MP4转换视频，然后使用affdex::VideoDetector detector(60, 1, affdex::FaceDetectorMode::SMALL_FACES);运行Affectiva时，Affectiva一直运行到最后并打印
“ [Affectiva]视频处理已完成”，但帧速率仅为23 fps。 Here是文件。
当我现在在此转换后的文件上使用affdex::VideoDetector detector(62, 1, affdex::FaceDetectorMode::SMALL_FACES);运行Affectiva时，Affectiva会在509秒后停止，并且不会打印“ [Affectiva]完善的视频处理”。 Here是文件。
答案 0 :(得分：1)
如果视频帧率为60，则使用大于60的数字来处理所有帧。 IIRC如果您仅使用61或62，则应该获得正确的帧数。