C.1 Statistical testing of receiver BER/BLER performance
25.1413GPPBase Station (BS) conformance testing (FDD)Release 17TS
C.1.1 Error Definition
Bit Error Ration (BER) and Block Error Ratio (BLER) are defined in section 3.1.
C.1.2 Test Method
Each test is performed in the following manner:
a) Setup the required test conditions.
b) Record the number of samples tested and the number of occurred events (bit error or block error)
c) Stop the test at a stop criterion which is minimum test time or an early pass or an early fail event.
d) Once the test is stopped decide according to the pass fail decision rules ( clause C.1.7)
C.1.3 Test Criteria
The test shall fulfil the following requirements:
a) good pass fail decision
1) to keep reasonably low the probability (risk) of passing a bad unit for each individual test;
2) to have high probability of passing a good unit for each individual test;
b) good balance between test time and statistical significance
3) to perform measurements with a high degree of statistical significance;
4) to keep the test time as low as possible.
C.1.4 Calculation assumptions
C.1.4.1 Statistical independence
a) It is assumed, that error events are rare (lim BER BLER 🡪 0) independent statistical events. However the memory of the convolutional /turbo coder is terminated after one TTI. Samples and errors are summed up every TTI. So the assumption of independent error events is justified.
b) In the BLER test with fading there is the memory of the multipath fading channel which interferes the statistical independence. A minimum test time is introduced to average fluctuations of the multipath fading channel. So the assumption of independent error events is justified approximately.
C.1.4.2 Applied formulas
The formulas, applied to describe the BER BLER test, are based on the following experiments:
1) After having observed a certain number of errors (ne) the number of samples are counted to calculate BER BLER. Provisions are made (note 1) such that the complementary experiment is valid as well:
2) After a certain number of samples (ns) the number of errors, occurred, are counted to calculate BER BLER.
Experiment 1) stipulates to use the following Chi Square Distribution with degree of freedom ne:
2*dchisq(2*NE,2*ne).
Experiment 2) stipulates to use the Poisson Distribution:
dpois(ne,NE)
(NE: mean of the distribution)
To determine the early stop conditions, the following inverse cumulative operation is applied:
0.5 * qchisq(D,2*ne). This is applicable for experiment (1) and (2).
D: wrong decision risk per test step
NOTE: Other inverse cumulative operations are available, however only this is suited for experiment (1) and (2).
C.1.4.3 Approximation of the distribution
The test procedure is as follows:
During a running measurement for a UE ns (number of samples) and ne (number of errors) are accumulated and from this the preliminary BER BLER is calculated. Then new samples up to the next error are taken. The entire past and the new samples are basis for the next preliminary BER BLER. Depending on the result at every step, the UE can pass, can fail or must continue the test.
As early pass- and early fail-UEs leave the statistical totality under consideration, the experimental conditions are changed every step resulting in a distribution that is truncated more and more towards the end of the entire test. Such a distribution can not any more be handled analytically. The unchanged distribution is used as an approximation to calculate the early fail and early pass bounds.
C.1.5 Definition of good pass fail decision.
This is defined by the probability of wrong decision F at the end of the test. The probability of a correct decision is 1-F.
The probability (risk) to fail a good DUT shall be ≤ F according to the following definition: The failed DUT is still better than the specified error ratio (Test requirement)with a probability of ≤ F.
The probability to pass a bad DUT shall be ≤ F according to the following definition: The passed DUT is still worse than M times the specified error ratio (M>1 is the bad DUT factor) with a probability of ≤ F.
This definitions lead to an early pass and an early fail limit:
Early fail: ber≥ berlimfail
(1)
For ne≥7
Early pass: ber ≤berlimbadpass
(2)
For ne ≥ 1
With
ber (normalized BER,BLER): BER,BLER according to C.1.1 divided by Test requirement
D: wrong decision probability for a test step . This is a numerically evaluated fraction of F, the wrong decision probability at the end of the test. See table C.1.
ne: Number of error events
M: bad DUT factor see table C.1.
qchisq: inverse-cumulative-function of the chi-squared-distribution
C.1.6 Good balance between test time and statistical significance
Three independent test parameters are introduced into the test and shown in Table C.1. These are the obvious basis of test time and statistical significance. From the first two of them four dependent test parameters are derived. The third independent test parameter is justified separately.
Table C.1: independent and dependent test parameters
|
Independent test parameters |
Dependent test parameters |
||||
|
Test Parameter |
Value |
Reference |
Test parameter |
Value |
Reference |
|
Bad DUT factor M |
1.5 |
Tables C.3 to C.9 |
Early pass/fail condition |
Curves |
Clause C.1.5 Figure C.1.9 |
|
Final probability of wrong pass/fail decision F |
0.2%, (0.02%, note 2) |
Clause C.1.5 |
Target number of error events |
345 |
Tables C.3 to C.9 |
|
Probability of wrong pass/fail decision per test step D |
0.0085%, (0.0008% and 0.008%, note 2) |
||||
|
Test limit factor TL |
1.234 |
Tables C.3 to C.9 |
|||
|
Minimum test time |
Table C.2 |
||||
The minimum test time is derived from the following justification:
1) For no propagation conditions and static propagation condition
No early fail calculated from fractional number of errors <1 (see note 1)
2) For multipath fading condition
No stop of the test until 990 wavelengths are crossed with the speed given in the fading profile.
3) For birth death propagation conditions
No stop of the test until 200 birth death transitions occur
4) For moving propagation conditions: 628 sec
This is necessary in order to pass all potential critical points in the moving propagation profile 4 times: Maximum rake window, Maximum adjustment speed, Intersection of moving taps
5) For high speed train conditions
Scenario 1: 82.3s. This corresponds to 4 complete cycles of approach towards and departure leave to and from a BS antenna
Scenario 2: The test shall continue until 990 wavelengths are crossed with the speed given in the fading profile (1.8s corresponding 300 km/h)
Scenario 3: 28.8s. This corresponds to 4 complete cycles of approach towards and departure from a BS antenna
Table C.2: minimum Test time
|
Fading profile |
Minimum test time |
|
Multipath propagation Case 1, Case 2 |
164 sec |
|
Multipath propagation Case 3 |
4.1 sec |
|
Multipath propagation Case 4 |
2 sec |
|
Birth Death propagation |
38.2 sec |
|
Moving propagation |
628 sec |
|
High speed train conditions Scenario 1 |
82.3 sec |
|
High speed train conditions Scenario 2 |
1.8 sec |
|
High speed train conditions Scenario 3 |
28.8 sec |
In table C.3 to C.9 the minimum test time is converted in minimum number of samples.
C.1.7 Pass fail decision rules
No decision is allowed before the minimum test time is elapsed.
1) If minimum Test time < time for target number of error events then the following applies: The required confidence level 1-F (= correct decision probability) shall be achieved. This is fulfilled at an early pass or early fail event.
For BER:
For every TTI (Transmit Time Interval) sum up the number of bits (ns) and the number if errors (ne) from the beginning of the test and calculate
BER1 (including the artificial error at the beginning of the test (Note 1))and
BER0 (excluding the artificial error at the beginning of the test (Note 1)).
If BER0 is above the early fail limit, fail the DUT.
If BER1 is below the early pass limit, pass the DUT.
Otherwise continue the test
For BLER:
For every TTI sum up the number of blocks (ns) and the number of erroneous blocks (ne) from the beginning of the test and calculate
BLER1 (including the artificial error at the beginning of the test (Note 1))and
BLER0 (excluding the artificial error at the beginning of the test (Note 1)).
If BLER1 is below the early pass limit, pass the DUT.
If BLER0 is above the early fail limit, fail the DUT.
Otherwise continue the test
2) If the minimum test time ≥ time for target error events, then the test runs for the minimum test time and the decision is done by comparing the result with the test limit.
For BER:
For every TTI (Transmit Time Interval) sum up the number of bits (ns) and the number if errors (ne) from the beginning of the test and calculate BER0
For BLER:
For every TTI sum up the number of blocks (ns) and the number of erroneous blocks (ne) from the beginning of the test and calculate BLER0
If BER0/BLER0 is above the test limit, fail the DUT.
If BER0/BLER0 is on or below the test limit, pass the DUT.
C.1.8 Test conditions for BER, BLER, Pd, E-DPCCH tests
Table C.3: Test conditions for BER tests
|
Type of test (BER) |
Propagation conditions |
Test requirement (BER) |
Test limit (BER)= Test requirement (BER)x TL TL |
Target number of error events (time) |
Minimum number of samples |
Prob that good unit will fail = Prob that bad unit will pass (%) |
Bad unit BER factor M |
|
Reference Sensitivity Level |
– |
0.001 |
1.234 |
345 (22.9s) |
Note 1 |
0.2 |
1.5 |
|
Dynamic Range |
– |
0.001 |
1.234 |
345 (22.9s) |
Note 1 |
0.2 |
1.5 |
|
Adjacent Channel Selectivity |
– |
0.001 |
1.234 |
345 (22.9s) |
Note 1 |
0.2 |
1.5 |
|
Blocking Characteristics Pass condition Note 2 |
– |
0.001 |
1.251 |
402 (26.3s) |
Note 1 |
0.2 |
1.5 |
|
Blocking Characteristics Fail condition Note 2 |
– |
0.001 |
1.251 |
402 (26.3s) |
Note 1 |
0.02 |
1.5 |
|
Intermodulation Characteristics |
– |
0.001 |
1.234 |
345 (22.9s) |
Note 1 |
0.2 |
1.5 |
|
Verification of internal BER calculation |
Not applicable, TS 34.121 Annex F.6.1.10 Dual limit BLER Tests may be applied in principle |
||||||
Table C.4: Test conditions for BLER tests
|
Type of test (BLER) |
Information Bit rate |
Test requirement (BLER) |
Test limit (BLER)= Test requirement (BLER)x TL TL |
Target number of error events (time) |
Minimum number of samples (time) |
Prob that bad unit will pass = Prob that good unit will fail (%) |
Bad unit BLER factor M |
|
Demodulation in Static Propagation conditions |
12.2 64 144 384 |
0.01 0.1 0.01 0.1 0.01 0.1 0.01 |
1.234 |
345 (559s) (112s) (1118s) (55.9s) (559s) (28s) (280s) |
Note 1 |
0.2 |
1.5 |
|
Demodulation of DCH in Multi-path Fading Propagation conditions (Case 1, Case 2) |
12.2 64 144 384 |
0.01 0.1 0.01 0.1 0.01 0.1 0.01 |
1.234 |
345 (559s) (112s) (1118s) (55.9s) (559s) (28s) (280s) |
(164s) 8200 4100 4100 8200 8200 16400 16400 |
0.2 |
1.5 |
|
Demodulation of DCH in Multi-path Fading Propagation conditions (Case3) |
12.2 64 144 384 |
0.01 0.001 0.1 0.01 0.001 0.1 0.01 0.001 0.1 0.01 0.001 |
1.234 |
345 (559s) (5592s) (112s) (1118s) (11183s) (55.9s) (559s) (5592s) (28s) (280s) (2796s) |
(4.1s) 205 205 103 103 103 205 205 205 410 410 410 |
0.2 |
1.5 |
|
Demodulation of DCH in Multi-path Fading Propagation conditions (Case 4) |
12.2 64 144 384 |
0.01 0.001 0.1 0.01 0.001 0.1 0.01 0.001 0.1 0.01 0.001 |
1.234 |
345 (559s) (5592s) (112s) (1118s) (11183s) (55.9s) (559s) (5592s) (28s) (280s) (2796s) |
(2s) 100 100 50 50 50 100 100 100 200 200 200 |
0.2 |
1.5 |
|
Demodulation of DCH in moving propagation conditions |
12.2 64 |
0.01 0.1 0.01 |
1.234 |
345 (559s) (112s) (1118s) |
(628s) 31400 15700 15700 |
0.2 |
1.5 |
|
Demodulation of DCH in birth/death propagation conditions |
12.2 64 |
0.01 0.1 0.01 |
1.234 |
345 (559s) (112s) (1118s) |
(38.2s) 1910 955 955 |
0.2 |
1.5 |
|
Demodulation of DCH in high speed train conditions |
12.2 |
0.01 |
1.234 |
345 (559s) |
Scenario 1 (82.3s) 4115 Scenario 2 (1.8s) 90 Scenario 3 (28.8s) 1440 |
0.2 |
1.5 |
|
Verification of internal BLER calculation |
Not applicable, TS 34.121 Annex F.6.1.10 Dual limit BLER Tests may be applied in principle |
||||||
Table C.5: Test conditions for Pd tests (Probability of detection)
|
Type of test |
Information Bit rate Not applicable |
Test requirement (1-Pd) |
Test limit (1-Pd)= Test requirement (1-Pd)x TL TL |
Target number of error events (time) |
Minimum number of samples (time) |
Prob that bad unit will pass = Prob that good unit will fail (%) |
Bad unit BLER factor M |
|
RACH preamble detection in static propagation conditions |
0.01 0.001 |
1.234 |
345 (29.8s) (298s) (net preamble TX time) |
Note 1 |
0.2 |
1.5 |
|
|
RACH preamble detection in multipath fading conditions (case3) |
0.01 0.001 |
1.234 |
345 (29.8s) (298s) (net preamble TX time) |
3844 preambles (4.1s) |
0.2 |
1.5 |
|
|
RACH preamble detection in high speed train conditions |
0.01 0.001 |
1.234 |
345 (29.8s) (298s) (net preamble TX time) |
Scenario 1 77157 preambles (82.3s) Scenario 2 1688 preambles (1.8s) Scenario 3 27000 preambles (28.8s) |
0.2 |
1.5 |
Table C.6: Test conditions for BLER tests
|
Type of test (BLER) |
Information Bits |
Test requirement (BLER) |
Test limit (BLER)= Test requirement (BLER)x TL TL |
Target number of error events (time) |
Minimum number of samples (time) |
Prob that bad unit will pass = Prob that good unit will fail (%) |
Bad unit BLER factor M |
|
Demodulation of RACH message in static propagation conditions |
168 bits 360 bits |
0.1 0.01 0.1 0.01 |
1.234 |
345 (55.9s) (559s) (55.9s) (559s) (net message TX time) |
Note 1 |
0.2 |
1.5 |
|
Demodulation of RACH message in multipath fading case 3 |
168 bits 360 bits |
0.1 0.01 0.1 0.01 |
1.234 |
345 55.9s) (559s) (55.9s) (559s) (net message TX time) |
205 messages (4.1s) |
0.2 |
1.5 |
|
Demodulation of RACH message in high speed train conditions |
168 bits 360 bits |
0.1 0.01 0.1 0.01 |
1.234 |
345 (55.9s) (559s) (55.9s) (559s) (net message TX time) |
Scenario 1 4115 messages (82.3s) Scenario 2 90 messages (1.8s) Scenario 3 1440 messages (28.8s) |
0.2 |
1.5 |
Table C.7: (void)
Table C.8: (void)
Table C.9: Test conditions for Error ratio tests
|
Type of test |
Information Bit rate (Not applicable) |
Test requirement error ratio |
Test limit (error ratio) = Test requirement (error rate) x TL TL |
Target number of error events (time) |
Minimum number of samples (time) |
Prob that bad unit will pass = Prob that good unit will fail (%) |
Bad unit Error ratio factor M |
|
ACK false alarm in static propagation conditions |
0.01 |
1.234 |
345 (18.6s) (net ACK/NACK TX time) |
Note 1 |
0.2 |
1.5 |
|
|
ACK false alarm in multipath fading conditions (Case 1, Case 2) |
0.01 |
1.234 |
345 (18.6s) (net ACK/NACK TX time) |
(164s) 246000 ACK/NAK slots |
0.2 |
1.5 |
|
|
ACK false alarm in multipath fading conditions (Case 3) |
0.01 |
1.234 |
345 (18.6s) (net ACK/NACK TX time) |
(4.1s) 6150 ACK/NAK slots |
0.2 |
1.5 |
|
|
ACK mis-detection in static propagation conditions |
0.01 |
1.234 |
345 (18.6s) (net ACK/NACK TX time) |
Note 1 |
0.2 |
1.5 |
|
|
ACK mis-detection in multipath fading conditions (Case 1, Case 2) |
0.01 |
1.234 |
345 (18.6s) (net ACK/NACK TX time) |
(164s) 246000 ACK/NAK slots |
0.2 |
1.5 |
|
|
ACK mis-detection in multipath fading conditions (Case 3) |
0.01 |
1.234 |
345 (18.6s) (net ACK/NACK TX time) |
(4.1s) 6150 ACK/NAK slots |
0.2 |
1.5 |
Table C.10: Test conditions E-DPCCH tests
|
Type of test |
Information Bit rate (Not applicable) |
Test requirement error ratio |
Test limit (error ratio) = Test requirement (error rate) x TL TL |
Target number of error events (time) |
Minimum number of samples (time) |
Prob that bad unit will pass = Prob that good unit will fail (%) |
Bad unit Error ratio factor M |
|
E-DPCCH false alarm in multipath fading conditions (PA3, PB3) |
0.01 |
1.234 |
345 (279.6s for10ms TTI) (55.9s for 2msTTI) |
(164s) 16400TTIs for 10msTTI, 82000 TTIs for 2ms TTI |
0.2 |
1.5 |
|
|
E-DPCCH false alarm in multipath fading conditions (VA30) |
0.01 |
1.234 |
345 (279.6s for10ms TTI) (55.9s for 2msTTI) |
(16.4s) 1640TTIs for 10msTTI, 8200 TTIs for 2ms TTI |
0.2 |
1.5 |
|
|
E-DPCCH false alarm in multipath fading conditions (VA120) |
0.01 |
1.234 |
345 (279.6s for10ms TTI) (55.9s for 2msTTI) |
(4.1s) 410TTIs for 10msTTI, 2050 TTIs for 2ms TTI |
0.2 |
1.5 |
|
|
E-DPCCH missed detection in multipath fading conditions (PA3,PB3) |
0.002 |
1.234 |
345 (1397.9s for 10ms TTI, 279.6s for 2ms TTI) |
(164s) 16400TTIs for 10msTTI, 82000 TTIs for 2ms TTI |
0.2 |
1.5 |
|
|
E-DPCCH missed detection in multipath fading conditions (VA30) |
0.002 |
1.234 |
345 (1397.9s for 10ms TTI, 279.6s for 2ms TTI) |
(16.4s) 1640TTIs for 10msTTI, 8200 TTIs for 2ms TTI |
0.2 |
1.5 |
|
|
E-DPCCH missed detection in multipath fading conditions (VA120) |
0.002 |
1.234 |
345 (1397.9s for 10ms TTI, 279.6s for 2ms TTI) |
(4.1s) 410TTIs for 10msTTI, 2050 TTIs for 2ms TTI |
0.2 |
1.5 |
C.1.9 Practical Use (informative)
See figure C.1.9:
The early fail limit represents formula (1) in C.1.5. The range of validity is ne≥ 7 ( ≥8 in case of blocking test) to ne =345
The early pass limit represents formula (2) in C.1.5. The range of validity is ne=1 to ne =345. See note 1
The intersection co-ordinates of both curves are : target number of errors ne = 345 and test limit TL = 1.234.
The range of validity for TL is ne>345.
A typical BER BLER test, calculated form the number of samples and errors (C.1.2.(b)) using experimental method (1) or (2) (see C.1.4.2 calculation assumptions) runs along the yellow trajectory. With an errorless sample the trajectory goes down vertically. With an erroneous sample it jumps up right. The tester checks if the BER BLER test intersects the early fail or early pass limits. The real time processing can be reduced by the following actions:
BLER0 (excluding the artificial error at the beginning of the test (Note 1)). is calculated only in case of an error event.
BER0 (excluding the artificial error at the beginning of the test (Note 1)). is calculated only in case of an error event within a TTI.
So the early fail limit cannot be missed by errorless samples.
The check against the early pass limit may be done by transforming formula (2) in C.1.5 such that the tester checks against a Limit-Number-of-samples ( NL(ne)) depending on the current number of errors (including the artificial error at the beginning of the test (Note 1)).
Early pass if
TR: test requirement (0.001)
Figure C.1.9
NOTE 1: At the beginning of the test, an artificial error is introduced. This ensures that an ideal DUT meets the valid range of the early pass limit. In addition this ensures that the complementary experiment (C.1.4.2 bullet point (2)) is applicable as well.
For the check against the early fail limit the artificial erroneous sample, introduced at the beginning of the test , is disregarded.
Due to the nature of the test, namely discrete error events, the early fail condition shall not be valid, when fractional errors <1 are used to calculate the early fail limit: Any early fail decision is postponed until number of errors ne ≥7. In the blocking test any early fail decision is postponed until number of errors ne ≥ 8.
NOTE 2: F=0.2% is intended to be used for a test containing a few BER/BLER tests (e.g. receiver sensitivity is repeated 12 times(3 RF Channels * 2 Power-supplies * 2 Temperatures). For a test containing many BER/BLER tests (e.g. blocking test) this value is not appropriate for a single BER/BLER test.
The blocking test contains approx. 12750 single BER tests. A DUT on the limit will fail approx. 25 to 26 times due to statistical reasons using wrong decision probability at the end of the test F= 0.2%. This shall be solved by the following rule:
All passes (based on F=0.2%) are accepted, including the wrong decisions due to statistical reasons.
An early fail limit based on F=0.02% instead of 0.2% is established. That ensures that wrong decisions due to statistical reasons are reduced to 2 to 3 in 12750 BER measurements. If the fail cases are ≤12, it is allowed to repeat each fail cases 1 time before the final verdict.
These asymmetric test conditions ensure that a DUT on the limit consumes hardly more test time for a blocking test than in the symmetric case and reduces the wrong decision probability considerably and on the other hand the repetition allowance sufficiently suppresses the residual statistically caused wrong verdict for the aggregate test.