5 Conformance
26.2693GPPConformance testingeCall data transferIn-band modem solutionTS
Conformance testing of the fixed-point IVS and PSAP transmitters shall be performed by demonstrating bit-exactness to the fixed-point reference C-code implementation (given in TS 26.268 [2]).
Bit-exactness means that given a specific input sequence, the corresponding digital output sequence of the component under test shall exactly match the time-aligned output sequence of the reference implementation [2] of that component. The delay of the output sequence shall not exceed a certain specified limit.
Conformance testing of the IVS and PSAP receiver implementations shall be performed by either demonstrating bit-exactness to the reference C-code, or by testing against a set of minimum performance requirements by means of objective measures. The bit-exact approach should be preferred over the application of objective measures if the implementation of the IVS and PSAP receivers follows the one given in the reference C-code.
The MSDs, codecs and channel conditions to be used in the testing for bit-exactness as well as for minimum performance requirements are further described in Annexes A and B. Annex B also specifies the procedures for evaluating the conformance of a receiver implementation with respect to the minimum performance requirements.
5.1 Bit-exactness
To guarantee interoperability and data integrity, the implementation of PSAP and IVS transmitters shall be bit-exact.
For the IVS and PSAP receivers, the conformance of the implementation may be tested by showing bit-exactness.
For bit-exact fixed-point implementations, test sequences and scripts are used for conformance testing. The test cases consist of input and output reference MSDs, control sequences, and PCM data files.
For testing transmitters, input control sequences are fed into the IVS/PSAP transmitter and the corresponding PCM data output of the IVS/PSAP transmitter is recorded. To meet the bit exactness criterion, all test cases must yield bit-exact results for the durations of the reference sequences, compared to the provided reference IVS/PSAP PCM output files, starting from the first non-zero output sample.
The maximum delay, measured as the number of zero-valued output samples of the IVS transmitter, as a reaction to any of the reference input sequences, shall not exceed 160 samples.
The maximum delay, measured as the number of zero-valued output samples of the PSAP transmitter, as a reaction to any of the reference input sequences, shall not exceed 320 samples.
For testing receivers, input PCM data files are fed into the IVS/PSAP receiver and the sequence of control states of the IVS/PSAP receiver is recorded after the processing of each received frame (duration 20 ms). To meet the conformance requirements, the control state sequence must be identical to the provided reference control sequence within a tolerable timing difference of one frame.
5.2 Minimum performance requirements
Objective measures are used for testing the fulfilment of minimum performance requirements. These measures are the same for all implementations. They shall be met by all non-bit exact fixed-point receiver implementations of the IVS and PSAP, and may also be used as an additional check for receiver implementations which have been shown to meet conformance by demonstrating bit-exactness.
The following minimum performance requirements apply to transmissions of single MSDs of length 140 bytes.
5.2.1 MSD transmission time
The average MSD transmission times serve as a performance indicator for the signal waveform demodulator and FEC decoder implementation. The MSD transmission time can be defined by five different metrics as described in Annex D of 3GPP TR 26.969 [3]:
1) As the interval from the time instant when the IVS transmitter writes the first non-zero sample onto the uplink channel until the MSD message has been correctly decoded at the PSAP. This metric was used during the initial eCall selection tests and is called "Figure-of-Merit" (FoM).
2) As the interval from the time instant of call connect state at PSAP side until the MSD message has been correctly decoded. This metric can assess the complete round-trip-time (RTT) of the PSAP in PUSH mode and is called PSAP-PUSH-RTT.
3) As the interval from the time instant when PSAP begins sending of first START message until the MSD message has been correctly decoded. This metric can assess the complete round-trip-time (RTT) of the PSAP in PULL mode and is called PSAP-PULL-RTT.
4) As the interval from the time instance when IVS detects call is connected until it detects the first HLACK message. This metric can assess the complete round-trip-time (RTT) of the PSAP in PUSH mode from IVS perspective and is called IVS-PUSH-RTT.
5) As the interval from the time instance when IVS detects the first START message from PSAP until it detects the first HLACK message. This metric can assess the complete round-trip-time (RTT) of the PSAP in PULL mode from IVS perspective and is called IVS-PULL-RTT.
The FoM metric for the full test campaign given in Annex B was also used in the eCall selection tests.. In the FoM, for each single transmission simulation, the transmission time is limited to a value of 200 s, i.e., if the MSD has not been correctly received after 200 s, the transmission attempt is aborted and the transmission time is counted as 200 s. The respective transmission attempt is recorded as a failure (and this constitutes a violation of the minimum performance requirement "Transmission failures" in clause 5.2.3).
All RTT metrics are suitable also for performance tests over-the-air or in lab environments as they require only measurement instances at either PSAP or IVS side. They do not require exact time synchronization between IVS and PSAP clocks as would be required for the FoM metric.
The following subsections refer to the minimum requirements for the FoM metric, which can be easily obtained through simulative assessment of the in-band modem implementation. Additional minimum requirements for the RTT metrics are listed in Annex C.
5.2.1.1 Full campaign (all AMR-FR codec modes plus GSM-FR)
For the specified full campaign given in Annex B, the average MSD transmission time (Figure of Merit) shall not exceed 2.90 s.
5.2.1.2 Error-free case for AMR 12.2 and FR
For the subset of error-free test cases in the AMR 12.2 and FR codecs of the full campaign, the average MSD transmission time shall not exceed 2.00 s.
5.2.1.3 GSM-HR codec
In the GSM-HR codec, the average MSD transmission time shall not exceed 18.00 s for the set of channel conditions comprising a C/I of 10, 7, and 4 dB as well as the error free case. 100 test cases per channel conditions shall be carried out.
5.2.1.4 Noisy conditions
For the specified full campaign in Annex B, the average MSD transmission time shall not exceed 3.20 s if white Gaussian noise (AWGN) of 10 dB SNR is added both to the PSAP transmitter output and at the PSAP receiver input signals.
Noise shall be added to the PCM data that is fed into the PSAP receiver and to the PCM data that is output from the PSAP transmitter. This reflects noise that could occur on an analogue line in a PSTN.
The noise power shall be set to yield 10 dB SNR. At the PSAP receiver, the reference signal power shall be obtained by averaging over all incoming signal intervals while the PSAP receiver is in NACK state, which is the case when it receives the data part of the MSD message. A separate reference signal power shall be computed for each codec and channel condition as an average of the 100 test cases of the attached official test campaign official_test_configuration_file.txt.
At the PSAP transmitter, the reference signal power shall be calculated by averaging over an entire feedback message [1] consisting of 20 frames.
For information, the standard deviations of the PSAP receiver input PCM signals are given in Table 1 for the reference implementation [2]. The average PSAP transmitter PCM output standard deviation is 2056.
Table 1: Standard deviations of PSAP input signals
|
Codec, |
Standard deviation |
Codec, |
Standard deviation |
|
Full Rate, 7 dB |
1223 |
AMR 7.95, 7 dB |
1067 |
|
Full Rate, 10 dB |
1190 |
AMR 7.95, 10 dB |
1059 |
|
Full Rate, 13 dB |
1167 |
AMR 7.40, 7 dB |
988 |
|
Full Rate, 16 dB |
1156 |
AMR 7.40, 10 dB |
971 |
|
Full Rate, clean |
1181 |
AMR 6.70, 7 dB |
954 |
|
Full Rate, RSSI |
1149 |
AMR 5.90, 4 dB |
931 |
|
AMR 12.20, 7 dB |
1112 |
AMR 5.90, 7 dB |
953 |
|
AMR 12.20, 10 dB |
1129 |
AMR 5.15, 4 dB |
836 |
|
AMR 12.20, 13 dB |
1092 |
AMR 5.15, 7 dB |
854 |
|
AMR 12.20, clean |
1130 |
AMR 4.75, 1 dB |
737 |
|
AMR 10.20, 7 dB |
1045 |
AMR 4.75, 4 dB |
762 |
|
AMR 10.20, 10 dB |
1052 |
AMR 4.75, 7 dB |
770 |
|
AMR 10.20, 13 dB |
994 |
AMR 4.75, RSSI |
771 |
The following C code fragment illustrates the application of noise (and a possible subsequent clipping) onto the PSAP input or output signals:
temp = (int) pcm + (int)(((double)sigma)*randn());
if (temp > 32767) {
noisy_pcm = 32767;
} else if (temp < -32768) {
noisy_pcm = -32768;
} else {
noisy_pcm = (short) temp;
}
5.2.1.5 Scaling of PCM signals
For the specified full campaign, the average MSD transmission times indicated in Table 2 shall not be exceeded for the given channel gains (caused, e.g., by different AGC settings), applied to the signals before the IVS and PSAP receivers.
Before being fed to the IVS and PSAP receivers, the signed 16-bit PCM data shall be amplified/attenuated by a constant gain factor and then be mapped on PCM values again. More specifically, the PCM values shall be multiplied with the amplitude scaling factors given in the second column of Table 2. Signal amplitudes exceeding the signed 16-bit PCM range (-32768 to +32767) shall be clipped. Testing shall be carried out with the gain factors in the range from -12 dB to +12 dB as given in Table 2. For the specified full campaign, the average MSD transmission times (FoM) shall not exceed the values given in the third column of Table 2 for the respective gains.
Table 2: Gains and maximum MSD transmission times for different scaling factors
|
Gain |
Amplitude scaling factor |
Maximum FoM |
|
-12 dB |
0.25 |
2.95 s |
|
-6 dB |
0.5 |
2.95 s |
|
+6 dB |
2.0 |
2.95 s |
|
+12 dB |
4.0 |
3.30 s |
The following C code fragment illustrates the scaling and a possible subsequent clipping of PCM signals:
pcmScaled = (double) pcm * (double) AGC_UL_SCALE_FACTOR;
if (pcmScaled >= 0.0) {
if (pcmScaled > (double) 32767)
pcm = 32767;
else
pcm = (short) (pcmScaled + 0.5);
} else {
if (pcmScaled < (double) -32768)
pcm = -32768;
else
pcm = (short) (pcmScaled – 0.5);
}
5.2.2 CRC check quality
The CRC on the uplink shall be evaluated. No incorrect MSD shall pass the CRC check in any test case of the specified full test campaign.
5.2.3 Transmission failures
No transmission failures shall occur with any test case of the full test campaign, i.e. no single MSD transmission time shall exceed 200 s.
5.2.4 False detection due to signalling tones
The IVS modem receiver shall not falsely detect eCall communication at any instance, when the tone test file (see Annex B) is used as modem input.
5.2.5 Push messages
This minimum performance requirement is only applicable for the case that IVS and PSAP in-band modems are configured to work in push mode.
It shall take a maximum of five SEND messages to trigger the PSAP eCall modem receiver in push mode. The test shall be carried out for the test conditions given in the official test campaign official_test_configuration_file.txt.
5.2.6 HLACK messages
The IVS receiver shall reliably detect a higher-layer ACK (HLACK) with no more than five transmitted HLACK messages. This test shall be carried out for the conditions given in the official test campaign. official_test_configuration_file.txt.