WebRTC + IOS + Freeswitch：无法听到音频

时间：2016-03-06 11:46:39

标签： ios webrtc voip freeswitch mod-verto

我试图在IOS上实施function（从iPhone调用桌面）。我使用mod_verto作为RTC方面，使用Google's libjingle library启动并运行。

  • 从我的iPhone拨打电话时，我使用this excellent tutorial（在本地计算机上下载并运行）在桌面浏览器上接听电话。
  • 在iPhone方面，我可以听到桌面上的音频，但是我在桌面端没有听到任何声音。

  • 如果我使用2个浏览器窗口（使用Verto Communicator）拨打电话，一切正常。

  • 完全披露 ，我使用ws://不安全的websocket连接到 FreeSWITCH的

这是我的JSONRPC日志：

发送登录请求： 

{"jsonrpc":"2.0","method":"login","id":1,"params":{"login":"1000@MY-IP-ADDRESS","loginParams":{},"userVariables":{},"passwd":"1234","sessid":"53FB0781-B586-4CDA-98C6-558680663B46"}}

登录响应： 

{"jsonrpc":"2.0","id":1,"result":{"message":"logged in","sessid":"53FB0781-B586-4CDA-98C6-558680663B46"}}

verto.invite（包括iPhone sdp）： 

{"jsonrpc":"2.0","method":"verto.invite","id":2,"params":{"dialogParams":{"remote_caller_id_number":"1008","useVideo":false,"useMic":"any","useStereo":false,"tag":"webcam","login":"1000@159.203.164.7","useCamera":"any","videoParams":{"minFrameRate":30,"minWidth":"1280","minHeight":"720"},"destination_number":"1008","screenShare":false,"caller_id_name":"FreeSWITCH User","caller_id_number":"1000","callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800","remote_caller_id_name":"Outbound Call","useSpeak":"any"},"sessid":"53FB0781-B586-4CDA-98C6-558680663B46","sdp":"v=0\r\no=- 8564086442942257834 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE audio video\r\na=msid-semantic: WMS\r\nm=audio 58157 UDP\/TLS\/RTP\/SAVPF 111 103 104 9 102 0 8 106 105 13 127 126\r\nc=IN IP4 82.166.93.197\r\na=rtcp:52576 IN IP4 82.166.93.197\r\na=candidate:3168280865 1 udp 2122260223 11.0.0.244 58157 typ host generation 0\r\na=candidate:1260196625 1 udp 2122194687 10.134.172.254 58951 typ host generation 0\r\na=candidate:3168280865 2 udp 2122260222 11.0.0.244 52576 typ host generation 0\r\na=candidate:1260196625 2 udp 2122194686 10.134.172.254 58945 typ host generation 0\r\na=candidate:4066106833 1 tcp 1518280447 11.0.0.244 60562 typ host tcptype passive generation 0\r\na=candidate:94302177 1 tcp 1518214911 10.134.172.254 60563 typ host tcptype passive generation 0\r\na=candidate:4066106833 2 tcp 1518280446 11.0.0.244 60564 typ host tcptype passive generation 0\r\na=candidate:94302177 2 tcp 1518214910 10.134.172.254 60565 typ host tcptype passive generation 0\r\na=candidate:1610196941 1 udp 1686052607 82.166.93.197 58157 typ srflx raddr 11.0.0.244 rport 58157 generation 0\r\na=candidate:1610196941 2 udp 1686052606 82.166.93.197 52576 typ srflx raddr 11.0.0.244 rport 52576 generation 0\r\na=candidate:2274372738 2 udp 1685987070 176.13.15.205 5834 typ srflx raddr 10.134.172.254 rport 58945 generation 0\r\na=candidate:2274372738 1 udp 1685987071 176.13.15.205 5840 typ srflx raddr 10.134.172.254 rport 58951 generation 0\r\na=ice-ufrag:g8lHDtPwH7m5xRex\r\na=ice-pwd:Q6jcBJNTWAyu0JTuIaQAeNI3\r\na=fingerprint:sha-256 0F:A1:68:51:87:3E:B4:C1:0D:33:97:40:78:22:2A:8C:D2:B6:46:23:F5:99:C9:88:5D:34:DB:E2:C5:94:B3:DD\r\na=setup:actpass\r\na=mid:audio\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:3 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/abs-send-time\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:111 opus\/48000\/2\r\na=fmtp:111 minptime=10; useinbandfec=1\r\na=rtpmap:103 ISAC\/16000\r\na=rtpmap:104 ISAC\/32000\r\na=rtpmap:9 G722\/8000\r\na=rtpmap:102 ILBC\/8000\r\na=rtpmap:0 PCMU\/8000\r\na=rtpmap:8 PCMA\/8000\r\na=rtpmap:106 CN\/32000\r\na=rtpmap:105 CN\/16000\r\na=rtpmap:13 CN\/8000\r\na=rtpmap:127 red\/8000\r\na=rtpmap:126 telephone-event\/8000\r\na=maxptime:60\r\nm=video 61966 UDP\/TLS\/RTP\/SAVPF 100 101 116 117 96\r\nc=IN IP4 82.166.93.197\r\na=rtcp:63816 IN IP4 82.166.93.197\r\na=candidate:3168280865 1 udp 2122260223 11.0.0.244 61966 typ host generation 0\r\na=candidate:1260196625 1 udp 2122194687 10.134.172.254 50435 typ host generation 0\r\na=candidate:3168280865 2 udp 2122260222 11.0.0.244 63816 typ host generation 0\r\na=candidate:1260196625 2 udp 2122194686 10.134.172.254 63396 typ host generation 0\r\na=candidate:4066106833 1 tcp 1518280447 11.0.0.244 60566 typ host tcptype passive generation 0\r\na=candidate:94302177 1 tcp 1518214911 10.134.172.254 60567 typ host tcptype passive generation 0\r\na=candidate:4066106833 2 tcp 1518280446 11.0.0.244 60568 typ host tcptype passive generation 0\r\na=candidate:94302177 2 tcp 1518214910 10.134.172.254 60569 typ host tcptype passive generation 0\r\na=candidate:1610196941 1 udp 1686052607 82.166.93.197 61966 typ srflx raddr 11.0.0.244 rport 61966 generation 0\r\na=candidate:1610196941 2 udp 1686052606 82.166.93.197 63816 typ srflx raddr 11.0.0.244 rport 63816 generation 0\r\na=candidate:2274372738 1 udp 1685987071 176.13.15.205 5879 typ srflx raddr 10.134.172.254 rport 50435 generation 0\r\na=candidate:2274372738 2 udp 1685987070 176.13.15.205 5860 typ srflx raddr 10.134.172.254 rport 63396 generation 0\r\na=ice-ufrag:g8lHDtPwH7m5xRex\r\na=ice-pwd:Q6jcBJNTWAyu0JTuIaQAeNI3\r\na=fingerprint:sha-256 0F:A1:68:51:87:3E:B4:C1:0D:33:97:40:78:22:2A:8C:D2:B6:46:23:F5:99:C9:88:5D:34:DB:E2:C5:94:B3:DD\r\na=setup:actpass\r\na=mid:video\r\na=extmap:2 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:3 http:\/\/www.webrtc.org\/experiments\/rtp-hdrext\/abs-send-time\r\na=extmap:4 urn:3gpp:video-orientation\r\na=recvonly\r\na=rtcp-mux\r\na=rtpmap:100 VP8\/90000\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtpmap:101 VP9\/90000\r\na=rtcp-fb:101 ccm fir\r\na=rtcp-fb:101 nack\r\na=rtcp-fb:101 nack pli\r\na=rtcp-fb:101 goog-remb\r\na=rtcp-fb:101 transport-cc\r\na=rtpmap:116 red\/90000\r\na=rtpmap:117 ulpfec\/90000\r\na=rtpmap:96 rtx\/90000\r\na=fmtp:96 apt=100\r\n"}}

致电创建的回复： 

{"jsonrpc":"2.0","id":2,"result":{"message":"CALL CREATED","callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800","sessid":"53FB0781-B586-4CDA-98C6-558680663B46"}}

调用verto.media： 

{"jsonrpc":"2.0","method":"verto.media","id":637,"params":{"sdp":"v=0\no=FreeSWITCH 1457232832 1457232833 IN IP4 159.203.164.7\ns=FreeSWITCH\nc=IN IP4 159.203.164.7\nt=0 0\na=msid-semantic: WMS TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub\nm=audio 30784 UDP/TLS/RTP/SAVPF 111 126\na=rtpmap:111 opus/48000/2\na=fmtp:111 useinbandfec=1; minptime=10\na=rtpmap:126 telephone-event/8000\na=silenceSupp:off - - - -\na=ptime:20\na=sendonly\na=fingerprint:sha-256 FE:CD:54:3E:2A:D7:DB:00:57:B7:D4:55:A8:EB:79:08:16:BB:B0:EA:43:44:42:9A:90:01:49:37:7B:31:48:F8\na=setup:active\na=rtcp-mux\na=rtcp:30784 IN IP4 159.203.164.7\na=ice-ufrag:qLh1zzclxONPNyQO\na=ice-pwd:G7g4Drkist37beYsP5jfvlqS\na=candidate:9922185636 1 udp 659136 159.203.164.7 30784 typ host generation 0\na=ssrc:1323504502 cname:bhqCyFkpPbjUPSk0\na=ssrc:1323504502 msid:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub a0\na=ssrc:1323504502 mslabel:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub\na=ssrc:1323504502 label:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Uba0\nm=video 31380 UDP/TLS/RTP/SAVPF 100\na=rtpmap:100 VP8/90000\na=sendonly\na=fingerprint:sha-256 FE:CD:54:3E:2A:D7:DB:00:57:B7:D4:55:A8:EB:79:08:16:BB:B0:EA:43:44:42:9A:90:01:49:37:7B:31:48:F8\na=setup:active\na=rtcp-mux\na=rtcp:31380 IN IP4 159.203.164.7\nb=AS:1024\na=rtcp-fb:100 ccm fir\na=rtcp-fb:100 nack\na=rtcp-fb:100 nack pli\na=ssrc:594893571 cname:bhqCyFkpPbjUPSk0\na=ssrc:594893571 msid:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub v0\na=ssrc:594893571 mslabel:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ub\na=ssrc:594893571 label:TcxpBqoS0j04fOIzkIArKYrlV7LCs9Ubv0\na=ice-ufrag:2KDK4wDMYuAuVdAZ\na=ice-pwd:YTpxObqpLuBEfig7TKHN6bqU\na=candidate:7508673635 1 udp 659136 159.203.164.7 31380 typ host generation 0\n","callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800"}}

调用了verto.answer： 

{"jsonrpc":"2.0","method":"verto.answer","id":638,"params":{"callID":"0CD433FC-A909-4DF2-BC46-0A4A94E9B800"}}

问：为了在浏览器端听到音频，我缺少什么？
任何信息表示赞赏：）

更新，添加了Verto Communicator

更新2 IOS：音频流代码 

...
let audioTrack = self.factory.audioTrackWithID("Local-Audio")
self.localMediaStream?.addAudioTrack(audioTrack);
self.peerConnection!.addStream(self.localMediaStream)
...

更新3 - 部分解决方案 在检查我的代码时，我发现用于向我的本地媒体流添加视频轨道的旧代码，禁用此部分解决了音频问题，但为什么？该代码有什么问题？

P.S Promise类是由朋友创建的，模仿JS Promise方法。

func getUserMedia(mediaOptions:Dictionary<String , Any>? = nil) -> Promise<RTCMediaStream>{
    return Promise<RTCMediaStream>(executor: { (resolve, reject) -> () in
        var cameraID:String?
        self.localMediaStream = self.factory.mediaStreamWithLabel("Local-Meida")

        //if video option is enabled (default true)

        //-------------- Disabling this section solves the audio issues --------------
        if(mediaOptions?["video"] as? Bool ?? true){
            for captureDevice in AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo){
                if (captureDevice.position == mediaOptions?["devicePosition"] as? AVCaptureDevicePosition ?? AVCaptureDevicePosition.Front){
                    cameraID = captureDevice.localizedName
                    break
                }
            }

            if(cameraID == nil){
                reject(NSError(domain: "No cammera detected", code: 0, userInfo: nil))
            }

            let capturer = RTCVideoCapturer.init(deviceName: cameraID)

            let videoSource = self.factory.videoSourceWithCapturer(capturer, constraints: mediaOptions?["constraints"] as? RTCMediaConstraints ?? nil)

            if let localVideoTrack = self.factory.videoTrackWithID("Local-Video", source: videoSource){
                //!!!! THIS IS THE PROBLEMATIC LINE !!!!
                self.localMediaStream?.addVideoTrack(localVideoTrack)
            }else{
                reject(NSError(domain: "No Video track", code: 0, userInfo: nil))
            }
        }
        //-------------- Disabling this section solves the audio issues --------------

        if(mediaOptions?["audio"] as? Bool ?? true){
            let audioTrack = self.factory.audioTrackWithID("Local-Audio")
            self.localMediaStream?.addAudioTrack(audioTrack);
        }
        self.peerConnection!.addStream(self.localMediaStream)

        resolve(self.localMediaStream!)
    })
}

在有问题的行调试 freeswitch log

1 个答案:

答案 0 :(得分：5)

更新

除非您检查媒体服务器，网络客户端和 iOS客户端，否则很难理解WebRTC实施的问题就我们而言。

您的案例是音频通话，因此您的 localStream 不需要包含视频流，但如果您仔细观察，则会看到 < em>您实际上是在移动信息流中添加了videoTrack ：

 if(mediaOptions?["video"] as? Bool ?? true){
        for captureDevice in AVCaptureDevice.devicesWithMediaType(AVMediaTypeVideo){
            if (captureDevice.position == mediaOptions?["devicePosition"] as? AVCaptureDevicePosition ?? AVCaptureDevicePosition.Front){
                cameraID = captureDevice.localizedName
                break
            }
        }

        if(cameraID == nil){
            reject(NSError(domain: "No cammera detected", code: 0, userInfo: nil))
        }

        let capturer = RTCVideoCapturer.init(deviceName: cameraID)

        let videoSource = self.factory.videoSourceWithCapturer(capturer, constraints: mediaOptions?["constraints"] as? RTCMediaConstraints ?? nil)

        if let localVideoTrack = self.factory.videoTrackWithID("Local-Video", source: videoSource){
            //!!!! THIS IS THE PROBLEMATIC LINE !!!!
            self.localMediaStream?.addVideoTrack(localVideoTrack)
        }else{
            reject(NSError(domain: "No Video track", code: 0, userInfo: nil))
        }
    }

导致问题的因素是：self.localMediaStream?.addVideoTrack(localVideoTrack)，因为您要将视频附加到 localStream 。

评论

正如我所提到的，我们可能会遇到不同的麻烦情况，在这里我根据我在构建类似系统时的经验列出一些意见：

  1. 您的MediaServer可能没有可以在成功状态下重定向和处理您的呼叫的实施，因为在您附加视频时还会添加其他内容（请参阅您的会话说明，实际发送的内容），它只是拒绝创建一个电话。
  2. 即使您的MediaServer处理了该方案，也会包含Client（桌面和移动）的正确实施方式发信号通知其协议。
  3. 您通过了所有测试，现在您正在添加视频和音频，因此您从移动设备启动localStream，同样需要以其他方式创建。然后，当您通过websockets添加流，删除流和其他内容时，您需要处理事件。

    4. 本例中的解决方案

    删除在 localStream 中添加 localTrack 的部分，然后即使您有错误，也不是由创建localStream引起的，所以此步骤目前已解决。< / p>

    原始答案

    这里我有一个我的工作版本，但根据您的需要进行调整，因为您只使用音频。

    创建和设置peerConnection（localSide）

    // Connecting to the socket
    .........

// Create PeerConnectionFactory
self.peerConnectionFactory = [[RTCPeerConnectionFactory alloc] init];

RTCMediaConstraints *constraints = [self defaultPeerConnectionConstraints];

// Initialize peerConnection based on a list of ICE Servers
self.peerConnection = [self.peerConnectionFactory peerConnectionWithICEServers:[self getICEServers] constraints:constraints delegate:self];

// Create the localStram which contains the audioTrack
RTCMediaStream *localStream = [self createLocalMediaStream];

// Add this stream to the peerConnection
[self.peerConnection addStream:localStream];

// Please be aware here that I am using blocks, as I created a wrapper for easier maintenance, but you can use createOfferWithDelegate: which will go back at your delegation
NSLog(@"Creating peer offer");
RTCManager *strongSelf = self;
[strongSelf.peerConnection createOfferWithCallback:^(RTCSessionDescription *sdp, NSError *error) {
    if (!error) {
        dispatch_async(dispatch_get_main_queue(), ^{
            NSLog(@"Success at creating offer, now setting local description");
            [strongSelf.peerConnection setLocalDescriptionWithCallback:^(NSError *error) {
                if (!error) {
                    dispatch_async(dispatch_get_main_queue(), ^{
                        NSLog(@"Success at setting local description");
                        // On my type of signalization here I am connected, but yours is based on what type of signalization requires
                    });
                }

            } sessionDescription:sdp];
        });
    }
} constraints:[strongSelf defaultPeerConnectionConstraints]];

    助手

    // Now here we create the stream which contains the audio (Please note the ID)
- (RTCMediaStream *)createLocalMediaStream {
    RTCMediaStream *localStream = [self.peerConnectionFactory mediaStreamWithLabel:@"ARDAMS"];
    [localStream addAudioTrack:[self.peerConnectionFactory audioTrackWithID:@"ARDAMSa0"]];

    return localStream;
}

- (RTCMediaConstraints *)defaultPeerConnectionConstraints {
    // DtlsSrtpKeyAgreement is required for Chrome and Firefox to interoperate.
    NSArray *optionalConstraints = @[[[RTCPair alloc] initWithKey:@"DtlsSrtpKeyAgreement" value:@"true"]];

    RTCMediaConstraints *constraints = [[RTCMediaConstraints alloc] initWithMandatoryConstraints:nil optionalConstraints:optionalConstraints];
    return constraints;
}

    请注意，您的问题可能也是因为没有在主线程上调用内容。

相关问题
最新问题