G.4 Statistical testing of Performance Requirements with BLER limit

38.521-43GPPNRPart 4: PerformanceRadio transmission and receptionRelease 17TSUser Equipment (UE) conformance specification

Tools: ARFCN - Frequency Conversion for 5G NR/LTE/UMTS/GSM

G.4.1 General

The test of receiver performance characteristics is twofold.

1. A signal or a combination of signals is offered to the RX port(s) of the receiver.

2. The ability of the receiver to demodulate /decode this signal is verified by measuring the throughput.

In (2) is the statistical aspect of the test and is treated here.

The minimum requirement for several receiver performance test cases is specified in regards of BLER: 0.001%.

G.4.2 Design of the test

The test is defined by the following design principles (see clause G.2, Theory):

1. The early decision concept is applied.

2. A second limit is introduced: Bad DUT factor M>1.

3. To decide the test pass:

Supplier risk is applied based on the Bad DUT quality

To decide the test fail:

Customer Risk is applied based on the specified DUT quality

G.4.3 Numerical definition of the pass fail limits for 0.001% BLER

The numerical pass/fail limit is derived by the following parameters:

1a) Limit Error Ratio = 0.001%

2a) Bad DUT factor M=1.5 (selectivity)

justification see: TS 34.121 Clause F.6.3.3

3) Confidence level CL = 99.999%

Table G.4.3-1: Pass fail limits

ne	ns_p	ns_f	ne	ns_p	ns_f	ne	ns_p	ns_f
0	1074532	1067	215	20006574	14871394	430	36441701	33298651
1	1074532	1067	216	20085020	14954177	431	36516711	33386452
2	1274645	1067	217	20163439	15036999	432	36591711	33474268
3	1444583	1067	218	20241831	15119861	433	36666702	33562097
4	1599072	4727	219	20320196	15202761	434	36741683	33649940
5	1743641	12160	220	20398535	15285701	435	36816654	33737797
6	1881111	23683	221	20476847	15368679	436	36891616	33825668
7	2013164	39190	222	20555133	15451695	437	36966568	33913553
8	2140902	58403	223	20633393	15534749	438	37041511	34001452
9	2265092	81000	224	20711628	15617841	439	37116445	34089364
10	2386297	106667	225	20789836	15700971	440	37191369	34177291
11	2504945	135116	226	20868019	15784137	441	37266283	34265231
12	2621369	166089	227	20946177	15867341	442	37341189	34353184
13	2735834	199360	228	21024309	15950581	443	37416085	34441151
14	2848557	234730	229	21102417	16033858	444	37490972	34529132
15	2959718	272025	230	21180499	16117172	445	37565849	34617126
16	3069467	311091	231	21258557	16200521	446	37640718	34705134
17	3177931	351792	232	21336590	16283906	447	37715577	34793155
18	3285220	394009	233	21414599	16367326	448	37790427	34881189
19	3391428	437636	234	21492584	16450782	449	37865268	34969237
20	3496637	482577	235	21570545	16534273	450	37940100	35057298
21	3600921	528746	236	21648482	16617799	451	38014923	35145372
22	3704343	576068	237	21726395	16701360	452	38089737	35233459
23	3806960	624473	238	21804284	16784955	453	38164542	35321560
24	3908823	673898	239	21882150	16868585	454	38239338	35409673
25	4009977	724286	240	21959993	16952248	455	38314125	35497800
26	4110465	775585	241	22037812	17035945	456	38388903	35585939
27	4210324	827748	242	22115608	17119676	457	38463672	35674092
28	4309587	880730	243	22193382	17203440	458	38538432	35762258
29	4408285	934492	244	22271133	17287238	459	38613184	35850436
30	4506448	988997	245	22348861	17371068	460	38687927	35938627
31	4604101	1044211	246	22426567	17454931	461	38762661	36026831
32	4701268	1100101	247	22504250	17538827	462	38837386	36115048
33	4797972	1156638	248	22581911	17622755	463	38912102	36203278
34	4894232	1213795	249	22659550	17706716	464	38986810	36291520
35	4990069	1271547	250	22737168	17790708	465	39061510	36379774
36	5085500	1329869	251	22814763	17874733	466	39136200	36468042
37	5180542	1388740	252	22892337	17958789	467	39210882	36556322
38	5275209	1448137	253	22969889	18042876	468	39285556	36644614
39	5369517	1508043	254	23047420	18126994	469	39360221	36732919
40	5463478	1568438	255	23124929	18211144	470	39434877	36821237
41	5557107	1629304	256	23202418	18295325	471	39509525	36909566
42	5650414	1690627	257	23279885	18379536	472	39584165	36997908
43	5743410	1752389	258	23357331	18463778	473	39658796	37086263
44	5836108	1814577	259	23434757	18548050	474	39733419	37174629
45	5928516	1877177	260	23512162	18632353	475	39808033	37263008
46	6020643	1940175	261	23589546	18716685	476	39882639	37351399
47	6112500	2003560	262	23666910	18801047	477	39957237	37439803
48	6204094	2067319	263	23744254	18885439	478	40031826	37528218
49	6295434	2131442	264	23821577	18969861	479	40106407	37616645
50	6386526	2195916	265	23898880	19054311	480	40180980	37705085
51	6477380	2260734	266	23976164	19138791	481	40255545	37793536
52	6568000	2325884	267	24053427	19223300	482	40330102	37882000
53	6658395	2391358	268	24130671	19307838	483	40404650	37970475
54	6748569	2457146	269	24207895	19392404	484	40479190	38058963
55	6838530	2523241	270	24285099	19476999	485	40553722	38147462
56	6928283	2589634	271	24362284	19561623	486	40628246	38235973
57	7017834	2656318	272	24439450	19646274	487	40702762	38324496
58	7107187	2723285	273	24516597	19730954	488	40777270	38413030
59	7196348	2790528	274	24593724	19815662	489	40851770	38501576
60	7285321	2858041	275	24670832	19900397	490	40926262	38590134
61	7374112	2925816	276	24747922	19985160	491	41000746	38678704
62	7462724	2993848	277	24824993	20069950	492	41075222	38767285
63	7551162	3062130	278	24902045	20154768	493	41149690	38855878
64	7639430	3130657	279	24979078	20239613	494	41224150	38944482
65	7727532	3199424	280	25056093	20324485	495	41298602	39033098
66	7815471	3268424	281	25133089	20409383	496	41373047	39121725
67	7903252	3337653	282	25210068	20494309	497	41447483	39210364
68	7990878	3407105	283	25287028	20579261	498	41521912	39299014
69	8078352	3476777	284	25363970	20664239	499	41596333	39387675
70	8165677	3546663	285	25440893	20749244	500	41670746	39476348
71	8252857	3616759	286	25517799	20834275	501	41745152	39565032
72	8339894	3687060	287	25594687	20919332	502	41819550	39653727
73	8426792	3757563	288	25671558	21004415	503	41893940	39742434
74	8513553	3828263	289	25748411	21089524	504	41968323	39831151
75	8600181	3899156	290	25825246	21174658	505	42042698	39919880
76	8686677	3970239	291	25902063	21259818	506	42117065	40008620
77	8773044	4041508	292	25978864	21345003	507	42191424	40097371
78	8859286	4112960	293	26055647	21430213	508	42265777	40186133
79	8945403	4184590	294	26132413	21515449	509	42340121	40274907
80	9031399	4256396	295	26209162	21600709	510	42414458	40363691
81	9117276	4328375	296	26285893	21685995	511	42488788	40452486
82	9203035	4400523	297	26362608	21771305	512	42563110	40541292
83	9288680	4472838	298	26439306	21856639	513	42637425	40630109
84	9374212	4545316	299	26515987	21941999	514	42711732	40718937
85	9459633	4617954	300	26592652	22027382	515	42786032	40807776
86	9544944	4690751	301	26669300	22112790	516	42860324	40896625
87	9630149	4763702	302	26745931	22198222	517	42934609	40985485
88	9715249	4836806	303	26822546	22283678	518	43008887	41074356
89	9800245	4910060	304	26899145	22369157	519	43083157	41163238
90	9885139	4983461	305	26975727	22454661	520	43157420	41252131
91	9969933	5057007	306	27052293	22540188	521	43231676	41341034
92	10054629	5130696	307	27128843	22625739	522	43305924	41429947
93	10139228	5204526	308	27205377	22711313	523	43380165	41518872
94	10223731	5278493	309	27281895	22796910	524	43454399	41607806
95	10308141	5352597	310	27358398	22882531	525	43528626	41696752
96	10392459	5426835	311	27434884	22968175	526	43602846	41785708
97	10476685	5501204	312	27511355	23053842	527	43677058	41874674
98	10560822	5575703	313	27587810	23139531	528	43751263	41963651
99	10644871	5650331	314	27664249	23225243	529	43825462	42052638
100	10728833	5725084	315	27740673	23310978	530	43899653	42141635
101	10812709	5799961	316	27817081	23396736	531	43973837	42230643
102	10896501	5874961	317	27893475	23482516	532	44048014	42319662
103	10980210	5950082	318	27969852	23568318	533	44122183	42408690
104	11063837	6025321	319	28046215	23654143	534	44196346	42497729
105	11147384	6100677	320	28122563	23739989	535	44270502	42586778
106	11230851	6176149	321	28198895	23825858	536	44344651	42675837
107	11314239	6251735	322	28275212	23911748	537	44418793	42764907
108	11397550	6327434	323	28351515	23997661	538	44492928	42853986
109	11480785	6403243	324	28427803	24083595	539	44567056	42943076
110	11563945	6479161	325	28504075	24169550	540	44641177	43032176
111	11647030	6555187	326	28580333	24255527	541	44715291	43121286
112	11730042	6631320	327	28656577	24341526	542	44789399	43210406
113	11812982	6707558	328	28732806	24427546	543	44863499	43299535
114	11895850	6783899	329	28809020	24513587	544	44937593	43388675
115	11978648	6860343	330	28885220	24599649	545	45011680	43477825
116	12061377	6936887	331	28961405	24685732	546	45085760	43566985
117	12144037	7013532	332	29037577	24771836	547	45159833	43656155
118	12226629	7090274	333	29113734	24857961	548	45233900	43745334
119	12309155	7167114	334	29189876	24944107	549	45307960	43834523
120	12391614	7244050	335	29266005	25030273	550	45382013	43923723
121	12474008	7321081	336	29342119	25116460	551	45456059	44012932
122	12556338	7398206	337	29418220	25202668	552	45530099	44102150
123	12638604	7475422	338	29494306	25288896	553	45604132	44191379
124	12720808	7552731	339	29570379	25375144	554	45678159	44280617
125	12802949	7630129	340	29646438	25461412	555	45752178	44369865
126	12885029	7707617	341	29722483	25547700	556	45826192	44459123
127	12967048	7785194	342	29798514	25634009	557	45900198	44548390
128	13049007	7862857	343	29874532	25720337	558	45974198	44637667
129	13130907	7940606	344	29950536	25806685	559	46048192	44726953
130	13212749	8018441	345	30026527	25893053	560	46122179	44816249
131	13294533	8096360	346	30102504	25979441	561	46196159	44905555
132	13376259	8174362	347	30178468	26065848	562	46270133	44994870
133	13457929	8252446	348	30254418	26152274	563	46344100	45084194
134	13539543	8330612	349	30330355	26238721	564	46418061	45173528
135	13621102	8408859	350	30406279	26325186	565	46492016	45262871
136	13702605	8487185	351	30482190	26411671	566	46565964	45352224
137	13784055	8565589	352	30558087	26498174	567	46639906	45441586
138	13865452	8644072	353	30633972	26584697	568	46713841	45530958
139	13946795	8722632	354	30709843	26671239	569	46787770	45620339
140	14028086	8801268	355	30785702	26757800	570	46861692	45709729
141	14109325	8879979	356	30861547	26844380	571	46935608	45799128
142	14190513	8958765	357	30937380	26930979	572	47009518	45888537
143	14271650	9037625	358	31013200	27017596	573	47083422	45977955
144	14352737	9116558	359	31089007	27104232	574	47157319	46067382
145	14433775	9195563	360	31164802	27190886	575	47231210	46156818
146	14514763	9274640	361	31240584	27277559	576	47305094	46246264
147	14595702	9353788	362	31316353	27364250	577	47378973	46335719
148	14676593	9433006	363	31392110	27450959	578	47452845	46425182
149	14757437	9512294	364	31467854	27537687	579	47526711	46514655
150	14838233	9591650	365	31543586	27624433	580	47600570	46604137
151	14918983	9671074	366	31619306	27711197	581	47674424	46693628
152	14999686	9750566	367	31695013	27797979	582	47748271	46783128
153	15080344	9830124	368	31770708	27884779	583	47822113	46872637
154	15160956	9909749	369	31846390	27971597	584	47895948	46962155
155	15241523	9989439	370	31922061	28058432	585	47969777	47051682
156	15322045	10069194	371	31997719	28145286	586	48043599	47141218
157	15402524	10149014	372	32073365	28232157	587	48117416	47230762
158	15482959	10228896	373	32149000	28319045	588	48191227	47320316
159	15563350	10308842	374	32224622	28405951	589	48265031	47409879
160	15643699	10388851	375	32300232	28492875	590	48338830	47499450
161	15724005	10468921	376	32375831	28579815	591	48412622	47589030
162	15804270	10549052	377	32451417	28666774	592	48486409	47678619
163	15884492	10629245	378	32526992	28753749	593	48560190	47768217
164	15964673	10709497	379	32602555	28840741	594	48633964	47857823
165	16044814	10789809	380	32678107	28927751	595	48707733	47947438
166	16124913	10870180	381	32753646	29014778	596	48781495	48037062
167	16204973	10950610	382	32829175	29101821	597	48855252	48126695
168	16284993	11031098	383	32904691	29188882	598	48929003	48216336
169	16364973	11111643	384	32980196	29275959	599	49002747	48305986
170	16444914	11192245	385	33055690	29363053	600	49076486	48395644
171	16524817	11272904	386	33131172	29450164	601	49150219	48485312
172	16604680	11353619	387	33206643	29537291	602	49223946	48574987
173	16684506	11434390	388	33282102	29624435	603	49297668	48664671
174	16764294	11515215	389	33357550	29711596	604	49371383	48754364
175	16844045	11596095	390	33432987	29798773	605	49445093	48844065
176	16923758	11677030	391	33508413	29885966	606	49518797	48933775
177	17003435	11758018	392	33583827	29973176	607	49592495	49023493
178	17083075	11839059	393	33659230	30060402	608	49666187	49113220
179	17162679	11920153	394	33734623	30147644	609	49739874	49202955
180	17242247	12001299	395	33810004	30234902	610	49813554	49292699
181	17321779	12082497	396	33885374	30322176	611	49887229	49382451
182	17401276	12163747	397	33960734	30409467	612	49960899	49472211
183	17480738	12245048	398	34036082	30496773	613	50034562	49561980
184	17560165	12326400	399	34111419	30584095	614	50108220	49651757
185	17639558	12407801	400	34186746	30671433	615	50181872	49741542
186	17718917	12489253	401	34262062	30758787	616	50255519	49831335
187	17798241	12570754	402	34337367	30846156	617	50329160	49921137
188	17877532	12652304	403	34412662	30933541	618	50402795	50010947
189	17956790	12733903	404	34487945	31020942	619	50476425	50100765
190	18036015	12815550	405	34563218	31108358	620	50550049	50190592
191	18115206	12897245	406	34638481	31195790	621	50623667	50280427
192	18194366	12978988	407	34713733	31283237	622	50697280	50370270
193	18273492	13060777	408	34788974	31370699	623	50770887	50460120
194	18352587	13142614	409	34864205	31458177	624	50844489	50549980
195	18431650	13224497	410	34939426	31545670	625	50918085	50639847
196	18510681	13306426	411	35014636	31633178	626	50991676	50729722
197	18589681	13388401	412	35089836	31720702	627	51065261	50819605
198	18668650	13470421	413	35165025	31808240	628	51138840	50909497
199	18747588	13552486	414	35240204	31895794	629	51212414	50999396
200	18826495	13634596	415	35315373	31983362	630	51285983	51089304
201	18905372	13716750	416	35390532	32070946	631	51359546	51179219
202	18984219	13798949	417	35465680	32158544	632	51433104	51269143
203	19063035	13881191	418	35540819	32246157	633	51506656	51359074
204	19141822	13963476	419	35615947	32333785	634	51580203	51449013
205	19220579	14045805	420	35691065	32421428	635	51653744	51538961
206	19299307	14128176	421	35766173	32509085	636	51727280	51628916
207	19378006	14210590	422	35841272	32596757	637	51800811	51718879
208	19456676	14293046	423	35916360	32684443	638	51874336	51808850
209	19535318	14375544	424	35991438	32772144	639	51947856	51898828
210	19613930	14458083	425	36066507	32859859	640	52021370	51988815
211	19692515	14540664	426	36141565	32947589	641	52094880	52078809
212	19771071	14623286	427	36216614	33035333	642	52168384	52168811
213	19849600	14705948	428	36291653	33123092
214	19928101	14788651	429	36366682	33210864

NOTE 1: The first column is the number of errors (ne = number of NACK).

NOTE 2: The second column is the number of samples for the pass limit (ns_p, ns=Number of Samples= number of NACK + ACK).

NOTE 3: The third column is the number of samples for the fail limit (ns_f).

NOTE 4: An ideal DUT passes after 1074532 samples. The maximum test time is 52171625 samples. A DUT passes, if the maximum number of samples is reached and it did not fail before.

G.4.4 Simulation to derive the pass-fail limits for 0.001% BLER

The binomial distribution and its inverse are used to design the pass and fail limits. Note that this method is not unique and that other methods exist.

Where

– fail(..) is the error ratio for the fail limit.

– pass(..) is the error ratio for the pass limit.

– ER is the specified error ratio 1e-5.

– ne is the number of bad results. This is the variable in both equations.

– M is the Bad DUT factor M=1.5.

– d_f is the wrong decision probability of a single (ne, ns) co-ordinate for the fail limit.
It is found by simulation to be d_f = 2e-7.

– cl_p is the confidence level of a single (ne, ns) co-ordinate for the pass limit.
It is found by simulation to be cl_p = 0.9999999.

– qnbinom(..): The inverse cumulative function of the negative binomial distribution.

The simulation works as follows:

– A large population of limit DUTs with true ER = 1e-5 is decided against the pass and fail limits.

– cl_p and d_f are tuned such that CL (99.999 %) of the population passes and D (0.001 %) of the population fails.

– A population of Bad DUTs with true ER = M*1e-5 is decided against the same pass and fail limits.

– cl_p and d_f are tuned such that CL (99.999 %) of the population fails and D (0.001 %) of the population passes.

– The number of DUTs decrease during the simulation, as the decided DUTs leave the population. That number decreases with an approximately exponential characteristics. After 642 bad results all DUTs of the population are decided.

NOTE: The exponential decrease of the population is an optimal design goal for the decision co-ordinates (ne, ns), which can be achieved with other formulas or methods as well.