C.2 Example criteria for video bit rate feedback and adaptation
26.1143GPPIP Multimedia Subsystem (IMS)Media handling and interactionMultimedia telephonyRelease 18TS
C.2.1 Introduction
This annex gives the outline of possible example adaptation criteria that make use of adaptation signalling for video as described in section 10.3. Several different adaptation implementations are possible and the example criteria shown in this section are not to be seen as an adaptive scheme excluding other designs. Implementers are free to use these example criteria or to use any other adaptation algorithm as long as the requirements and recommendations specified in clause 10.3 are fulfilled. The description of the example criteria is split into two parts, one for the media sender side and one for the media receiver side.
C.2.2 Media sender side
The basic rate adaptation algorithm on the media sender side serves to combine the received RTCP TMMBR and RTCP Receiver Report or Sender Report in a way that makes adaptation possible in the presence of any or both of the aforementioned reports. One important aspect is that the TMMBR reports will serve as an upper limit on the permitted bitrate while RTCP Receiver or Sender Reports provide a means for temporary adjustments based on e.g. the packet loss rate in a given interval. Note that the actual bitrate limit will also depend on the bandwidth attribute in the SDP. Typically adjustments of the permitted bitrate due to TMMBR reports are less frequent than adjustments due to RTCP Receiver or Sender Reports The media sender may use the following conditions to adjust its video transmission rate:
1 Upon receiving a TMMBR message the media sender sets its maximum transmission rate to the requested rate.
2 If the requested rate in the TMMBR message is greater than the current video transmission rate the media sender (gradually) increases its transmission rate to the requested rate in the TMMBR message.
3 Examining the RTCP Receiver Report and Sender Report information to determine whether finer adaptations can be made to the video transmission rate. For example, the media sender may reduce its video transmission rate in response to an increase in the packet loss rate.
4 Reducing the video transmission rate if the media sender determines that the local radio uplink throughput is decreasing, e.g. detecting congestion in the uplink transmission buffers or examining other indicators of uplink quality.
5 Other conditions
C.2.3 Media receiver side
The basic rate adaptation algorithm on the media receiver side may consist of both the well-established RFC 3550 RTCP RR and SR reporting (which is not described further) and the estimation and sending of TMMBR to the sender. Transmission of TMMBR reports is typically less frequent than RTCP Receiver Reports or Sender Reports.
The media receiver may use the following conditions to send a TMMBR message requesting a reduction in video transmission rate
1 The video packets are arriving too close to or too late for their scheduled playout
2 The receiver detects an unacceptably high packet loss rate
3 The receiver detects that the received bitrate has been reduced
4 Other conditions
The MTSI media receiver may use the following conditions to send a TMMBR requesting an increase in video transmission rate
1 The video packets are arriving earlier than needed for their scheduled playout
2 Other conditions
Care must be taken when sending consecutive TMMBR messages to accommodate the media sender’s reaction to previously sent TMMBR messages. When doing this, the media receiver should account for delays in the transmission of TMMBR messages due to RTCP bandwidth requirements.
C.2.4 Video encoder bitrate adaptation, down-switch
As described in clause 10.3.4.2, the video encoder may not be able to immediately change the sending bitrate to the requested bitrate, especially if this also requires changing the frame rate and/or the video resolution. During this period, the generated bitrate is higher than the channel capacity which means that excessive bits (excess_bits) are generated and the corresponding RTP packets will be queuing up at the node where the congestion occurs. Figure C.8 gives a simplified schematic description of the encoder bitrate adaptation. The encoder bitrate is here shown with straight lines. In reality, the amount of data generated by the encoder will vary from frame to frame and the bitrate will then vary around the bitrate shown in this figure. These bitrate variations are not considered in this description but needs to be considered in the real implementation.
The encoder uses the bitrate of the previous channel capacity (prev_rate) as long as the sender is unaware of the reduced channel capacity. When the TMMBR message is received, the encoder starts reducing the bitrate towards the new channel capacity (new_rate). Since the bitrate indicated in the TMMBR message includes the IP/UDP/RTP overhead then this needs to be removed. The encoder bitrate is then reduced even further for a while to gain back the delay caused by the queue build-up.
Figure C.8: Schematic figure of bitrate reduction in video encoder when the encoder cannot immediately switch to the requested bitrate
The upper limit requirement () for how many excessive bits that is allowed to be generated during the adaptation time is defined by the Worst-Case Adaptation Algorithm, which corresponds to the rectangle
while the recommendation (
) is defined by the triangle
.
The amount of excessive bits that are generated corresponds to the area of the triangle . As can be seen in the figure, the measurement window (
) is here longer than the worst case adaptation time (
) which is used for defining the requirement limit and the recommendation limit for the excessive bits. In this example, the encoder bitrate is not reduced fast enough to fulfil the recommendation but the requirement is fulfilled. This shows that it is allowed to use an adaptation time that is longer than the time used for defining the requirement and recommendation, as long as the amount of excessive bits does not exceed the limit.
After the encoder bitrate has been reduced to the new bitrate then the encoder needs to reduce the bitrate even further to form the beginning of the delay recovery period. The length of the delay recovery period depends on the amount of excessive bits that have been generated and how much lower the encoding bitrate is compared to the new channel capacity.
In reality, the queue starts to build up even before the sender has received the TMMBR message, which is here shown by the rectangle . The sender can estimate the excessive bits generated during this period from the RTCP Receiver Reports and can then compensate for this by extending the delay recovery period.
C.2.5 Video encoder bitrate adaptation, receiver-driven up-switch
When the channel is being under-utilized by the sender, it is likely that the delivery of video packets will occur before they actually need to be played out at the receiver. Therefore, the sender rate could be increased and introduce some additional delay without negatively affecting the system or the user experience.
The excess bits () that can be introduced into the transmission path can be computed as follows in the case that the channel bandwidth is equal to the average receiving rate measured at the receiver, i.e., the worst case with no spare channel bandwidth available:
(C.2.5-1)
where: is the bitrate with which the sending rate is increased;
is the Round-Trip Time;
and: is the time needed to detect if congestion occurs, see Figure C.8.
The corresponding worst case excess delay () due to excess_bits equals:
(C.2.5-2)
where: is the average throughput as measured by the receiver.
Therefore, if the receiver determines the amount of allowable excess delay () from the received video packets, it can calculate the amount of rate increase that would not congest the system as:
(C.2.5-3)
Since the one-way delay from sender to receiver is generally unknown to the receiver, it cannot use this to calculate the allowable excess delay. Instead the receiver measures the amount of time between when a packet arrives and when it is scheduled to be played out to determine how much additional delay is acceptable. This metric is actually more accurate from a user-experience perspective since this directly determines whether the video information in received packets can actually be displayed to the user without degradation.
The bitrate to request () in TMMBR is then:
(C.2.5-4)
Before sending the TMMBR message with the requested rate, the receiver needs to verify that the requested rate does not exceed the bitrate that was negotiated in SDP offer/answer.
C.2.6 Video encoder bitrate adaptation, recovery phase
The delay recovery rate () and delay recovery period (
) can be determined as follows. Let
(C.2.6-1)
where: (
) is the rate undershoot factor and may depend on the magnitude of the channel rate drop (
) is:
(C.2.6-2)
For example, if is large, then
may be proportionally large or if
is small, then
may be proportionally small, for example:
(C.2.6-3)
Assuming that the bits during time period are queued up and are contributing to the delay, the delay recovery period
can be computed as follows:
(C.2.6-4)
A minimum bit rate requirement for the encoder may exist that applies to and, therefore, also to
as follows:
(C.2.6-5)
and therefore:
(C.2.6-6)
with:
(C.2.6-7)
If during the delay recovery period a TMMBR message is received that carries a new rate value (), and
is significantly larger than
, for example
, then the delay recovery period may be shortened. Conversely, if
then an extended delay recovery period can be computed.
Annex D (informative):
Reference delay computation algorithm
In this annex, the reference jitter management algorithm is described. It is written in pseudo code and is non-causal; hence non-implementable. The purpose of this algorithm is to define an "ideal" behaviour which all jitter buffers used in MTSI should strive to mimic. This buffer operates based on three input parameters:
– lookback factor to set the current target buffering depth;
– target late loss rate;
– maximum allowed time scaling percentage.
function ref_jb(channel,jb_adaptation_lookback,delay_delta_max,target_loss)
% channel = file name of the channel
% lookback = look back factor when estimating the max jitter
% buffer level [number of frames]
% delay_delta_max = max timescaling related modification (%) of the
% delay
% target_loss = target late loss (%)
% example syntax:
% ref_jb(‘channel_1.dat’,200,15,0.5);
framelength = 20;
% this value sets the speech data in each RTP packet to 20 ms. For 2 speech
% frames/RTP packet the value would be 40 ms.
jitter_est_window=50;
% Sets the jitter estimation window in number of frames
delay_delta_max_ms = framelength*delay_delta_max*0.01;
% Sets the maximum allowed time scaling
tscale = 1;
% Scale factor of delay data
% In this case the files are assumend to be ascii files with one delay
% entry per line, the entries are in ms, a negative value denotes
% a packet loss.
x = load(channel);
x =x’;
% remove packet losses
% remove inital startup empty frames
ix = find(x > 0);
x(1:ix(1)-1) = x(ix(1));
% remove packet losses (replace with nearby delay values)
ix = find(x < 0);
packet_loss = length(ix)/length(x)*100;
for n=1:length(ix)
if (ix(n) > 1)
x(ix(n)) = x(ix(n)-1);
end;
end;
% convert timescale to ms
x = x*tscale;
L = length(x);
T = 1:L;
% estimate min and max TX delay, estimate a delta_delay
for n=1:L
ix = [max(1,n-jitter_est_window):n];
max_delay(n) = max(x(ix));
min_delay(n) = min(x(ix));
delta_delay(n) = max_delay(n)-min_delay(n);
end
% compute the target max jitter buffer level with some slow adaptation
% downwards, just to mimick how a jitter buffer might behave
for n=1:L
ix = [max(1,n-jb_adaptation_lookback):n];
jb(n) = max(delta_delay(ix));
% The timescaling is not allowed to adjust the jitterbuffer target max level
% too fast.
if n == 1
jb_ = jb(n);
end
delta = abs(jb_-jb(n));
if delta < delay_delta_max_ms;
jb_ = jb(n);
else
if (jb(n) < jb_)
jb_ = jb_-delay_delta_max_ms;
else
jb_ = jb_+delay_delta_max_ms;
end
jb(n) = jb_;
end
% jitter buffer target max level can only assume an integer number of frames
jbq(n) = ceil(jb(n)/framelength)*framelength;
% compute estimated delay
del(n) = jbq(n)+min_delay(n);
end
if target_loss > 0
% decrease the max jitter buffer leve until a target late loss has been
% reached.
late_loss = length(find(del < x))/L*100.0;
jbq_save = jbq; % as the max level is increased until the late loss > target one
% must be able to revert back to the previous data
while late_loss < target_loss
jbq_save = jbq;
jbq = min(max(jbq)-framelength,jbq);
del = jbq+min_delay;
late_loss = length(find(del < x))/L*100.0;
end
jbq = jbq_save;
del = jbq+min_delay;
end
jdel = max(0,del-x);
%Calculate and plot the CDF of the reference buffer.
figure(1);plot(T,jbq,T,del,T,x);
[n,x] = hist(jdel,140); y = cumsum(n);y = y/max(y)*100;
figure(2);plot(x,y);axis([0 200 0 100]);ylabel(‘%’);xlabel(‘ms’);title(‘CDF of packet delay in JB’);
Annex E (informative):
QoS profiles