Question

我使用node.js从WebRTC服务器（我使用mediasoup）获取RTP流，并从流中获取解密的RTP数据包原始数据。我想将此RTP数据转发到ffmpeg，然后我可以将其保存到文件，或将其作为RTMP流推送到其他媒体服务器。我想最好的方法是创建描述音频和视频流的SDP文件，并通过新的套接字发送数据包。

ffmpeg命令是：

ffmpeg -loglevel debug -protocol_whitelist file,crypto,udp,rtp -re -vcodec libvpx -acodec opus -i test.sdp -vcodec libx264 -acodec aac -y output.mp4

我尝试通过UDP发送数据包：

v=0
o=mediasoup 7199daf55e496b370e36cd1d25b1ef5b9dff6858 0 IN IP4 192.168.193.182
s=7199daf55e496b370e36cd1d25b1ef5b9dff6858
c=IN IP4 192.168.193.182
t=0 0
m=audio 33301 RTP/AVP 111
a=rtpmap:111 /opus/48000
a=fmtp:111 minptime=10;useinbandfec=1
a=rtcp-fb:111 transport-cc
a=sendrecv
m=video 33302 RTP/AVP 100
a=rtpmap:100 /VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=sendrecv

但我总是得到（删除无聊的部分）：

Opening an input file: test.sdp.

[sdp @ 0x103dea0]
Format sdp probed with size=2048 and score=50
[sdp @ 0x103dea0] audio codec set to: (null)
[sdp @ 0x103dea0] audio samplerate set to: 44100
[sdp @ 0x103dea0] audio channels set to: 1
[sdp @ 0x103dea0] video codec set to: (null)
[udp @ 0x10402e0] end receive buffer size reported is 131072
[udp @ 0x10400c0] end receive buffer size reported is 131072
[sdp @ 0x103dea0] setting jitter buffer size to 500
[udp @ 0x1040740] bind failed: Address already in use
[AVIOContext @ 0x1046980] Statistics: 473 bytes read, 0 seeks
test.sdp: Invalid data found when processing input

请注意，即使我根本不打开套接字或向此端口发送任何内容，我也会得到它，就像ffmpeg本身尝试多次打开这些端口一样。

我还尝试打开两个（视频和音频）TCP服务器并使用TCP定义SDP：

v=0
o=mediasoup 7199daf55e496b370e36cd1d25b1ef5b9dff6858 0 IN IP4 192.168.193.182
s=7199daf55e496b370e36cd1d25b1ef5b9dff6858
c=IN IP4 192.168.193.182
t=0 0
m=audio 33301 TCP 111
a=rtpmap:111 /opus/48000
a=fmtp:111 minptime=10;useinbandfec=1
a=rtcp-fb:111 transport-cc
a=setup:active
a=connection:new
a=sendrecv
m=video 33302 TCP 100
a=rtpmap:100 /VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=setup:active
a=connection:new
a=sendrecv

但是我没有看到任何传入连接到我的TCP服务器，我从ffmpeg得到以下内容：

Opening an input file: test.sdp.

[sdp @ 0xdddea0]
Format sdp probed with size=2048 and score=50

[sdp @ 0xdddea0]
audio codec set to: (null)

[sdp @ 0xdddea0]
audio samplerate set to: 44100
[sdp @ 0xdddea0] audio channels set to: 1
[sdp @ 0xdddea0] video codec set to: (null)
[udp @ 0xde02e0] end receive buffer size reported is 131072
[udp @ 0xde00c0] end receive buffer size reported is 131072
[sdp @ 0xdddea0] setting jitter buffer size to 500
[udp @ 0xde0740] end receive buffer size reported is 131072

[udp @ 0xde0180] end receive buffer size reported is 131072
[sdp @ 0xdddea0] setting jitter buffer size to 500
[sdp @ 0xdddea0] Before avformat_find_stream_info() pos: 593 bytes read:593 seeks:0 nb_streams:2
[libvpx @ 0xdeea80] v1.3.0
[libvpx @ 0xdeea80] --target=x86_64-linux-gcc --enable-pic --disable-install-srcs --as=nasm --enable-shared --prefix=/usr --libdir=/usr/lib64

[sdp @ 0xdddea0] Could not find codec parameters for stream 1 (Video: vp8, 1 reference frame, none): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[sdp @ 0xdddea0] After avformat_find_stream_info() pos: 593 bytes read:593 seeks:0 frames:0
Input #0, sdp, from 'test.sdp':
  Metadata:
    title           : 7199daf55e496b370e36cd1d25b1ef5b9dff6858
  Duration: N/A, bitrate: N/A
    Stream #0:0, 0, 1/90000: Audio: opus, 48000 Hz, mono, fltp
    Stream #0:1, 0, 1/90000: Video: vp8, 1 reference frame, none, 90k tbr, 90k tbn, 90k tbc
Successfully opened the file.
Parsing a group of options: output file output.mp4.
Successfully parsed a group of options.
Opening an output file: output.mp4.
[file @ 0xde3660] Setting default whitelist 'file,crypto'
Successfully opened the file.

detected 1 logical cores
[graph 0 input from stream 0:0 @ 0xde3940] Setting 'time_base' to value '1/48000'
[graph 0 input from stream 0:0 @ 0xde3940] Setting 'sample_rate' to value '48000'
[graph 0 input from stream 0:0 @ 0xde3940] Setting 'sample_fmt' to value 'fltp'
[graph 0 input from stream 0:0 @ 0xde3940] Setting 'channel_layout' to value '0x4'
[graph 0 input from stream 0:0 @ 0xde3940] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x4
[audio format for output stream 0:0 @ 0xe37900] Setting 'sample_fmts' to value 'fltp'
[audio format for output stream 0:0 @ 0xe37900] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
[AVFilterGraph @ 0xde0220] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed

Output #0, mp4, to 'output.mp4':

  Metadata:

    title           :
7199daf55e496b370e36cd1d25b1ef5b9dff6858


    encoder         :
Lavf57.56.100


    Stream #0:0
, 0, 1/48000
: Audio: aac (LC) ([64][0][0][0] / 0x0040), 48000 Hz, mono, fltp, delay 1024, 69 kb/s


    Metadata:

      encoder         :
Lavc57.64.100 aac


Stream mapping:

  Stream #0:0 -> #0:0 (opus (native) -> aac (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)

test.sdp: Connection timed out
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[output stream 0:0 @ 0xde3b40] EOF on sink link output stream 0:0:default.
No more output streams to write to, finishing.
[aac @ 0xde2b00] Trying to remove 1024 samples, but the queue is empty
[aac @ 0xde2b00] Trying to remove 1024 more samples than there are in the queue
[mp4 @ 0xe6a540] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[mp4 @ 0xe6a540] Encoder did not produce proper pts, making some up.
[aac @ 0xde2b00] Trying to remove 1024 samples, but the queue is empty
[aac @ 0xde2b00] Trying to remove 1024 more samples than there are in the queue
size=       1kB time=00:00:00.04 bitrate= 157.9kbits/s speed=0.00426x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3268.000000%
Input file #0 (test.sdp):
  Input stream #0:0 (audio): 0 packets read (0 bytes); 0 frames decoded (0 samples);
  Input stream #0:1 (video): 0 packets read (0 bytes);
  Total: 0 packets (0 bytes) demuxed
Output file #0 (output.mp4):
  Output stream #0:0 (audio): 0 frames encoded (0 samples); 2 packets muxed (25 bytes);
  Total: 2 packets (25 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0xde37a0] Statistics: 30 seeks, 25 writeouts
[aac @ 0xde2b00] Qavg: 47249.418

[AVIOContext @ 0xde6980] Statistics: 593 bytes read, 0 seeks

请注意上面日志中的“连接超时”。

我猜我的SDP都错了，有什么建议吗？

SDP的替代方案也受到欢迎。

Answer 1

c=IN IP4 192.168.193.182

这是您自己的Node UDP / TCP服务器从ffmpeg监听连接的本地IP吗？

m=audio 33301 RTP/AVP 111

为什么选择33301？我希望这不是与mediasoup用来与远程浏览器通信的相同的端口（如果是这样，显然你会得到“已经在使用的地址”错误）...

a=rtpmap:111 /opus/48000

这是错误的格式。删除第一个“/".

删除所有a = rtcp-fb行。我认为ffmpeg根本不支持任何一种。

视频相同。

Answer 2

我解决了我的问题，我的设置：

exlpck.SaveAs(ms)

问题出在sdp中。 sdp应该是正确的格式，如sdp的答案。在文件（ffmpeg sdp输入文件）和媒体服务器（在我的情况下为kms）中使用相同的sdp内容。 IP应该是nginx服务器ip。（在我的情况下适用）。

Answer 3

对于 RTP/UDP，当 ffmpeg 绑定到端口 33301 时，它将自动绑定到 +1 端口，在这种情况下为 33302，对于 RTCP，这就是为什么当它尝试使用时你会收到绑定失败错误33302 视频。

使用SDP将RTP流式传输到FFMPEG

3 个答案: