webrtc帶寬預測---發送端預測

SR RR

通過RR包,將丟包率信息回傳給發送端,根據丟包率做發送端帶寬估計。接收RTCP包,以及對應處理流程基本相同,具體流程如下:

void UdpTransportImpl::IncomingRTCPCallback
void UdpTransportImpl::IncomingRTCPFunction
void VideoChannelTransport::IncomingRTCPPacket
int ViENetworkImpl::ReceivedRTCPPacket
int32_t ViEChannel::ReceivedRTCPPacket
int ViEReceiver::ReceivedRTCPPacket
int ViEReceiver::InsertRTCPPacket
int32_t ModuleRtpRtcpImpl::IncomingRtcpPacket
void RTCPReceiver::TriggerCallbacksFromRTCPPacket
void OnReceivedRtcpReceiverReport
void BitrateControllerImpl::OnReceivedRtcpReceiverReport
void SendSideBandwidthEstimation::UpdateReceiverBlock
void SendSideBandwidthEstimation::UpdateEstimate

bitrate_controller模塊接口是bitrate_controller_impl.h文件,通過這個文件就可以建立整個模塊。

int ViEBaseImpl::CreateChannel
int ViEBaseImpl::CreateChannel
rtc::scoped_ptr<ChannelGroup> group
ChannelGroup::ChannelGroup
BitrateController* BitrateController::CreateBitrateController

其他的對象也都在此BitrateControllerImpl類中,從而將整個模塊聯系起來。

有一個線程一直會輪詢這個更新帶寬評估的函數,因為接收到反饋包之后,會進行評估,這里具體什么用意,后面看。

int32_t BitrateControllerImpl::Process()
void SendSideBandwidthEstimation::UpdateEstimate(int64_t now_ms)

設置保留帶寬,用于音頻

int Conductor::VideoSetStream
int ViERTP_RTCPImpl::SetReservedTransmitBitrate
bool ViEChannelManager::SetReservedTransmitBitrate
void BitrateControllerImpl::SetReservedBitrate

獲取用于編碼器的起始碼率

int Conductor::VideoSetSendCodec
int ViECodecImpl::SetSendCodec
int32_t ViEEncoder::SetEncoder
bool BitrateControllerImpl::AvailableBandwidth

根據上一步的起始碼率以及設置的起始碼率,取較大值設置到SendSideBandwidthEstimation

int32_t ViEEncoder::SetEncoder
void BitrateControllerImpl::SetStartBitrate
void SendSideBandwidthEstimation::SetSendBitrate

設置最小最大碼率,這個碼率怎么來的,應該是上面設置下來的,后面看。

int32_t ViEEncoder::SetEncoder
void BitrateControllerImpl::SetMinMaxBitrate
void SendSideBandwidthEstimation::SetMinMaxBitrate

設置模式,然后決定是否使用RembSuppressor,這個做什么用,后面看。

int32_t ViEEncoder::SetEncoder
void BitrateControllerImpl::SetCodecMode(webrtc::VideoCodecMode mode) 
void RembSuppressor::SetEnabled(bool enabled)

計算RTT

照抄下面這篇博客,驗證基本正確。

http://www.cnblogs.com/lingdhox/p/5746210.html

A 發送 SR 包, 并記錄SR包的發送時間. 記為send_time
B 接收到 A的SR包后, 記錄下最后一次接受到SR包的時間. 記為last_recv_time... (B等待發送rtcp包)B 發送 RR包, 計算從[last_recv_time] 到 當前時間的延時. 記錄為delay_since_last_SR. 附加到RR包中. A 收到 B的RR包后, 計算RTT

RTT = send_time - delay_since_last_SR - last_recv_time
A -> 發送SR包.
ModuleRtpRtcpImpl::Process
RTCPSender::SendRTCP
RTCPSender::PrepareRTCP
RTCPSender::BuildSR

PrepareRTCP 中通過_sending(是否是發送端) 狀態判定發送SR或RR. SR包中含有發送時的NTP時間戳. BuildSR中_lastSendReport 記錄NTP時間的中間32位. 可以標識SR包, 也就是B回應RR包中report block的LSR字段(last SR timestamp ), 通過LSR可以查找_lastRTCPTime._lastRTCPTime記錄RTCP_NUMBER_OF_SR個數的SR發送時間.這兩個數組是一一對應的.

_lastRTCPTime[0] = Clock::NtpToMs(NTPsec, NTPfrac);
_lastSendReport[0] = (NTPsec << 16) + (NTPfrac >> 16);

最后SendToNetwork.

B -> 接收到SR包.
ModuleRtpRtcpImpl::IncomingRtcpPacket
RTCPReceiver::IncomingRTCPPacket
RTCPReceiver::HandleSenderReceiverReport

在HandleSenderReceiverReport 中保存 SR包中的NTP時間戳

_remoteSenderInfo.NTPseconds = rtcpPacket.SR.NTPMostSignificant;
_remoteSenderInfo.NTPfraction = rtcpPacket.SR.NTPLeastSignificant;

并記錄SR包接到時的NTP時間戳

_clock->CurrentNtp(_lastReceivedSRNTPsecs, _lastReceivedSRNTPfrac);
B -> 發送RR包

獲取回饋狀態, 并發送給A

ModuleRtpRtcpImpl::Process()
if (rtcp_sender_.TimeToSendRTCPReport()) {
rtcp_sender_.SendRTCP(GetFeedbackState(), kRtcpReport);
}
ModuleRtpRtcpImpl::GetFeedbackState()
ModuleRtpRtcpImpl::LastReceivedNTP

state.last_rr_ntp_secs 和state.last_rr_ntp_frac即為上一次接收到SR包時, 記錄的_clock->CurrentNtp(_lastReceivedSRNTPsecs, _lastReceivedSRNTPfrac); 時間戳state.remote_sr 通過_remoteSenderInfo.NTPseconds 和 _remoteSenderInfo.NTPfraction, 取中間32位算出.

RTCPSender::PrepareReport
在這里計算延時, 填充到report block中.

// get our NTP as late as possible to avoid a race
_clock->CurrentNtp(*ntp_secs, *ntp_frac);

// Delay since last received report
uint32_t delaySinceLastReceivedSR = 0;
if ((feedback_state.last_rr_ntp_secs != 0) ||
(feedback_state.last_rr_ntp_frac != 0)) {
// get the 16 lowest bits of seconds and the 16 higest bits of fractions
uint32_t now=*ntp_secs&0x0000FFFF;
now <<=16;
now += (*ntp_frac&0xffff0000)>>16;

uint32_t receiveTime = feedback_state.last_rr_ntp_secs&0x0000FFFF;
receiveTime <<=16;
receiveTime += (feedback_state.last_rr_ntp_frac&0xffff0000)>>16;

delaySinceLastReceivedSR = now-receiveTime;
}
report_block->delaySinceLastSR = delaySinceLastReceivedSR;
report_block->lastSR = feedback_state.remote_sr;

report_block->delaySinceLastSR 即為 從接到SR包到發送RR包之間的延時.
report_block->lastSR 即SR包中NTP時間戳的中間32位. (在A端_lastSendReport數組中記錄).

A 收到 B的RR包
ModuleRtpRtcpImpl::IncomingRtcpPacket
RTCPReceiver::IncomingRTCPPacket
RTCPReceiver::HandleSenderReceiverReport
RTCPReceiver::HandleReportBlock

通過 lastSR 到sender模塊中取出SR包的發送時間.

uint32_t sendTimeMS =
_rtpRtcp.SendTimeOfSendReport(rtcpPacket.ReportBlockItem.LastSR);

計算RTT .

uint32_t delaySinceLastSendReport =
rtcpPacket.ReportBlockItem.DelayLastSR;

// local NTP time when we received this
uint32_t lastReceivedRRNTPsecs = 0;
uint32_t lastReceivedRRNTPfrac = 0;

_clock->CurrentNtp(lastReceivedRRNTPsecs, lastReceivedRRNTPfrac);

// time when we received this in MS
uint32_t receiveTimeMS = Clock::NtpToMs(lastReceivedRRNTPsecs,
lastReceivedRRNTPfrac);

// Estimate RTT
uint32_t d = (delaySinceLastSendReport & 0x0000ffff) * 1000;
d /= 65536;
d += ((delaySinceLastSendReport & 0xffff0000) >> 16) * 1000;

int32_t RTT = 0;

if (sendTimeMS > 0) {
RTT = receiveTimeMS - d - sendTimeMS;
....
}

注意:
delay since last SR (DLSR) 的單位是1/65536秒.
為什么在獲取本地時間不直接獲取,而要先得到NTP再轉為毫秒,應該是要時間統一都用NTP時間,而不是本地時間。

NTP相關計算

通過_clock->CurrentNtp()方法來獲得當前時刻的ntp,其中lastReceivedRRNTPsecs 為秒,lastReceivedRRNTPfrac 為小數點后面部分。

  // local NTP time when we received this
  uint32_t lastReceivedRRNTPsecs = 0;
  uint32_t lastReceivedRRNTPfrac = 0;
  _clock->CurrentNtp(lastReceivedRRNTPsecs, lastReceivedRRNTPfrac);
  // time when we received this in MS
  void CurrentNtp(uint32_t& seconds, uint32_t& fractions) const override {
    timeval tv = CurrentTimeVal();
    double microseconds_in_seconds;
    Adjust(tv, &seconds, &microseconds_in_seconds);
    fractions = static_cast<uint32_t>(
        microseconds_in_seconds * kMagicNtpFractionalUnit + 0.5);
  }

通過CurrentTimeVal()方法的復雜計算之后,最后得到秒和微妙。

  timeval CurrentTimeVal() const override {
    const uint64_t FILETIME_1970 = 0x019db1ded53e8000;

    FILETIME StartTime;
    uint64_t Time;
    struct timeval tv;

    // We can't use query performance counter since they can change depending on
    // speed stepping.
    GetTime(&StartTime);

    Time = (((uint64_t) StartTime.dwHighDateTime) << 32) +
           (uint64_t) StartTime.dwLowDateTime;

    // Convert the hecto-nano second time to tv format.
    Time -= FILETIME_1970;

    tv.tv_sec = (uint32_t)(Time / (uint64_t)10000000);
    tv.tv_usec = (uint32_t)((Time % (uint64_t)10000000) / 10);
    return tv;
  }

Adjust()對秒和微秒做微調。

  static void Adjust(const timeval& tv, uint32_t* adjusted_s,
                     double* adjusted_us_in_s) {
    *adjusted_s = tv.tv_sec + kNtpJan1970;
    *adjusted_us_in_s = tv.tv_usec / 1e6;

    if (*adjusted_us_in_s >= 1) {
      *adjusted_us_in_s -= 1;
      ++*adjusted_s;
    } else if (*adjusted_us_in_s < -1) {
      *adjusted_us_in_s += 1;
      --*adjusted_s;
    }
  }
};

通過NtpToMs()將NTP時間轉為毫秒

  int64_t receiveTimeMS = Clock::NtpToMs(lastReceivedRRNTPsecs,
                                         lastReceivedRRNTPfrac);
int64_t Clock::NtpToMs(uint32_t ntp_secs, uint32_t ntp_frac) {
  const double ntp_frac_ms = static_cast<double>(ntp_frac) / kNtpFracPerMs;
  return 1000 * static_cast<int64_t>(ntp_secs) +
      static_cast<int64_t>(ntp_frac_ms + 0.5);
}
最后編輯于
?著作權歸作者所有,轉載或內容合作請聯系作者
平臺聲明:文章內容(如有圖片或視頻亦包括在內)由作者上傳并發布,文章內容僅代表作者本人觀點,簡書系信息發布平臺,僅提供信息存儲服務。

推薦閱讀更多精彩內容

  • RTSP SDP RTP/RTCP 介紹應用層 RTSP、SDP; 傳輸層 RTP、TCP、UDP; 網絡層 IP...
    Atom_Woo閱讀 3,902評論 0 7
  • 音視頻同步事關多媒體產品的最直觀用戶體驗,是音視頻媒體數據傳輸和渲染播放的最基本質量保證。音視頻如果不同步,有可能...
    weizhenwei閱讀 13,122評論 7 18
  • 音視頻同步事關多媒體產品的最直觀用戶體驗,是音視頻媒體數據傳輸和渲染播放的最基本質量保證。音視頻如果不同步,有可能...
    Monktan閱讀 2,291評論 1 8
  • 一 前言 RF3550定義實時傳輸協議RTP和它的控制協議RTCP。RTP協議是Internet上針對流媒體傳輸的...
    weizhenwei閱讀 15,990評論 5 31
  • Spring Cloud為開發人員提供了快速構建分布式系統中一些常見模式的工具(例如配置管理,服務發現,斷路器,智...
    卡卡羅2017閱讀 134,981評論 19 139