D.4 Pass fail decisions
37.571-13GPPPart 1: Conformance test specificationRelease 16TSUser Equipment (UE) conformance specification for UE positioning
D.4.1 Numerical definition of the pass fail limits for A-GNSS test cases
Table D.4.1: Pass/fail limits for A-GNSS test cases
ne |
nsp |
nsf |
ne |
nsp |
nsf |
ne |
nsp |
nsf |
ne |
nsp |
nsf |
0 |
77 |
NA |
43 |
855 |
576 |
86 |
1525 |
1297 |
129 |
2173 |
2050 |
1 |
106 |
NA |
44 |
871 |
592 |
87 |
1540 |
1314 |
130 |
2188 |
2067 |
2 |
131 |
NA |
45 |
887 |
608 |
88 |
1556 |
1331 |
131 |
2203 |
2085 |
3 |
154 |
NA |
46 |
903 |
625 |
89 |
1571 |
1349 |
132 |
2218 |
2103 |
4 |
176 |
NA |
47 |
919 |
641 |
90 |
1586 |
1366 |
133 |
2233 |
2121 |
5 |
197 |
NA |
48 |
935 |
657 |
91 |
1601 |
1383 |
134 |
2248 |
2139 |
6 |
218 |
42 |
49 |
951 |
674 |
92 |
1617 |
1401 |
135 |
2263 |
2156 |
7 |
238 |
52 |
50 |
967 |
690 |
93 |
1632 |
1418 |
136 |
2277 |
2174 |
8 |
257 |
64 |
51 |
982 |
706 |
94 |
1647 |
1435 |
137 |
2292 |
2192 |
9 |
277 |
75 |
52 |
998 |
723 |
95 |
1662 |
1453 |
138 |
2307 |
2210 |
10 |
295 |
87 |
53 |
1014 |
739 |
96 |
1677 |
1470 |
139 |
2322 |
2227 |
11 |
314 |
100 |
54 |
1030 |
756 |
97 |
1692 |
1487 |
140 |
2337 |
2245 |
12 |
333 |
112 |
55 |
1046 |
772 |
98 |
1708 |
1505 |
141 |
2352 |
2263 |
13 |
351 |
125 |
56 |
1061 |
789 |
99 |
1723 |
1522 |
142 |
2367 |
2281 |
14 |
369 |
139 |
57 |
1077 |
805 |
100 |
1738 |
1540 |
143 |
2381 |
2299 |
15 |
387 |
152 |
58 |
1093 |
822 |
101 |
1753 |
1557 |
144 |
2396 |
2317 |
16 |
405 |
166 |
59 |
1108 |
839 |
102 |
1768 |
1574 |
145 |
2411 |
2335 |
17 |
422 |
180 |
60 |
1124 |
855 |
103 |
1783 |
1592 |
146 |
2426 |
2352 |
18 |
440 |
194 |
61 |
1140 |
872 |
104 |
1798 |
1609 |
147 |
2441 |
2370 |
19 |
457 |
208 |
62 |
1155 |
889 |
105 |
1813 |
1627 |
148 |
2456 |
2388 |
20 |
474 |
222 |
63 |
1171 |
906 |
106 |
1828 |
1644 |
149 |
2470 |
2406 |
21 |
492 |
237 |
64 |
1186 |
922 |
107 |
1844 |
1662 |
150 |
2485 |
2424 |
22 |
509 |
251 |
65 |
1202 |
939 |
108 |
1859 |
1679 |
151 |
2500 |
2442 |
23 |
526 |
266 |
66 |
1217 |
956 |
109 |
1874 |
1697 |
152 |
2515 |
2460 |
24 |
543 |
281 |
67 |
1233 |
973 |
110 |
1889 |
1714 |
153 |
2530 |
2478 |
25 |
560 |
295 |
68 |
1248 |
990 |
111 |
1904 |
1732 |
154 |
2544 |
2496 |
26 |
577 |
310 |
69 |
1264 |
1007 |
112 |
1919 |
1750 |
155 |
2559 |
2513 |
27 |
593 |
325 |
70 |
1279 |
1024 |
113 |
1934 |
1767 |
156 |
2574 |
2531 |
28 |
610 |
341 |
71 |
1295 |
1040 |
114 |
1949 |
1785 |
157 |
2589 |
2549 |
29 |
627 |
356 |
72 |
1310 |
1057 |
115 |
1964 |
1802 |
158 |
2603 |
2567 |
30 |
643 |
371 |
73 |
1326 |
1074 |
116 |
1979 |
1820 |
159 |
2618 |
2585 |
31 |
660 |
387 |
74 |
1341 |
1091 |
117 |
1994 |
1838 |
160 |
2633 |
2603 |
32 |
676 |
402 |
75 |
1357 |
1108 |
118 |
2009 |
1855 |
161 |
2648 |
2621 |
33 |
693 |
418 |
76 |
1372 |
1126 |
119 |
2024 |
1873 |
162 |
2662 |
2639 |
34 |
709 |
433 |
77 |
1387 |
1143 |
120 |
2039 |
1890 |
163 |
2677 |
2657 |
35 |
725 |
449 |
78 |
1403 |
1160 |
121 |
2054 |
1908 |
164 |
2692 |
2675 |
36 |
742 |
465 |
79 |
1418 |
1177 |
122 |
2069 |
1926 |
165 |
2707 |
2693 |
37 |
758 |
480 |
80 |
1433 |
1194 |
123 |
2084 |
1943 |
166 |
2721 |
2711 |
38 |
774 |
496 |
81 |
1449 |
1211 |
124 |
2099 |
1961 |
167 |
2736 |
2729 |
39 |
790 |
512 |
82 |
1464 |
1228 |
125 |
2114 |
1979 |
168 |
2751 |
2747 |
40 |
807 |
528 |
83 |
1479 |
1245 |
126 |
2128 |
1997 |
169 |
2765 |
NA |
41 |
823 |
544 |
84 |
1495 |
1263 |
127 |
2143 |
2014 |
|||
42 |
839 |
560 |
85 |
1510 |
1280 |
128 |
2158 |
2032 |
NOTE: The first column is the number of bad results (ne)
The second column is the number of results for the pass limit (nsp)
The third column is the number of results for the fail limit (nsf)
D.4.2 Pass fail decision rules for A-GNSS test cases
Having observed 0 bad results, pass the test at ≥77 results, otherwise continue
Having observed 1 bad result, pass the test at ≥106 results, otherwise continue
Having observed 2 bad results, pass the test at ≥131 results, otherwise continue
etc. until
Having observed 6 bad results, pass the test at ≥218 results, fail the test at ≤ 42 results, otherwise continue
Having observed 7 bad results, pass the test at ≥238 results, fail the test at ≤ 52 results, otherwise continue
etc. until
Having observed 168 bad results, pass the test at ≥2751 results, fail the test at ≤2747 results, otherwise continue
Having observed 169 bad results, pass the test at ≥2765 results, otherwise fail
NOTE: an ideal DUT passes after 77 results. The maximum test time is 2765 results.
D.4.3 Numerical definition of the pass fail limits for ECID, OTDOA, MBS, WLAN and BLE test cases
Table D.4.3: Pass/fail limits for ECID, OTDOA, MBS, WLAN and BLE test cases
ne |
nsp |
nsf |
ne |
nsp |
nsf |
ne |
nsp |
nsf |
ne |
nsp |
nsf |
0 |
33 |
NA |
43 |
408 |
283 |
86 |
737 |
644 |
129 |
1056 |
1021 |
1 |
46 |
NA |
44 |
416 |
291 |
87 |
745 |
653 |
130 |
1064 |
1030 |
2 |
58 |
2 |
45 |
424 |
299 |
88 |
752 |
661 |
131 |
1071 |
1039 |
3 |
69 |
5 |
46 |
432 |
307 |
89 |
760 |
670 |
132 |
1078 |
1048 |
4 |
79 |
8 |
47 |
440 |
315 |
90 |
767 |
679 |
133 |
1086 |
1057 |
5 |
89 |
12 |
48 |
447 |
324 |
91 |
775 |
687 |
134 |
1093 |
1066 |
6 |
99 |
17 |
49 |
455 |
332 |
92 |
782 |
696 |
135 |
1100 |
1074 |
7 |
109 |
22 |
50 |
463 |
340 |
93 |
790 |
705 |
136 |
1108 |
1083 |
8 |
118 |
27 |
51 |
471 |
348 |
94 |
797 |
713 |
137 |
1115 |
1092 |
9 |
127 |
33 |
52 |
478 |
356 |
95 |
804 |
722 |
138 |
1122 |
1101 |
10 |
136 |
39 |
53 |
486 |
365 |
96 |
812 |
731 |
139 |
1130 |
1110 |
11 |
145 |
45 |
54 |
494 |
373 |
97 |
819 |
739 |
140 |
1137 |
1119 |
12 |
154 |
51 |
55 |
502 |
381 |
98 |
827 |
748 |
141 |
1144 |
1128 |
13 |
163 |
58 |
56 |
509 |
389 |
99 |
834 |
757 |
142 |
1152 |
1137 |
14 |
172 |
64 |
57 |
517 |
398 |
100 |
842 |
766 |
143 |
1159 |
1147 |
15 |
180 |
71 |
58 |
525 |
406 |
101 |
849 |
774 |
144 |
1166 |
1155 |
16 |
189 |
78 |
59 |
532 |
414 |
102 |
857 |
783 |
145 |
1174 |
1164 |
17 |
197 |
85 |
60 |
540 |
423 |
103 |
864 |
792 |
146 |
1181 |
1173 |
18 |
206 |
92 |
61 |
548 |
431 |
104 |
871 |
801 |
147 |
NA |
1182 |
19 |
214 |
99 |
62 |
555 |
440 |
105 |
879 |
809 |
148 |
||
20 |
223 |
106 |
63 |
563 |
448 |
106 |
886 |
818 |
149 |
||
21 |
231 |
113 |
64 |
571 |
456 |
107 |
894 |
827 |
150 |
||
22 |
239 |
120 |
65 |
578 |
465 |
108 |
901 |
836 |
151 |
||
23 |
248 |
128 |
66 |
586 |
473 |
109 |
909 |
844 |
152 |
||
24 |
256 |
135 |
67 |
594 |
482 |
110 |
916 |
853 |
153 |
||
25 |
264 |
142 |
68 |
601 |
490 |
111 |
923 |
862 |
154 |
||
26 |
272 |
150 |
69 |
609 |
499 |
112 |
931 |
871 |
155 |
||
27 |
281 |
157 |
70 |
616 |
507 |
113 |
938 |
880 |
156 |
||
28 |
289 |
165 |
71 |
624 |
516 |
114 |
946 |
888 |
157 |
||
29 |
297 |
173 |
72 |
632 |
524 |
115 |
953 |
897 |
158 |
||
30 |
305 |
180 |
73 |
639 |
533 |
116 |
960 |
906 |
159 |
||
31 |
313 |
188 |
74 |
647 |
541 |
117 |
968 |
915 |
160 |
||
32 |
321 |
196 |
75 |
654 |
550 |
118 |
975 |
924 |
161 |
||
33 |
329 |
204 |
76 |
662 |
558 |
119 |
983 |
933 |
162 |
||
34 |
337 |
211 |
77 |
669 |
567 |
120 |
990 |
941 |
163 |
||
35 |
345 |
219 |
78 |
677 |
575 |
121 |
997 |
950 |
164 |
||
36 |
353 |
227 |
79 |
684 |
584 |
122 |
1005 |
959 |
165 |
||
37 |
361 |
235 |
80 |
692 |
592 |
123 |
1012 |
968 |
166 |
||
38 |
369 |
243 |
81 |
700 |
601 |
124 |
1019 |
977 |
167 |
||
39 |
377 |
251 |
82 |
707 |
610 |
125 |
1027 |
986 |
168 |
||
40 |
385 |
259 |
83 |
715 |
618 |
126 |
1034 |
994 |
169 |
||
41 |
393 |
267 |
84 |
722 |
627 |
127 |
1042 |
1003 |
|||
42 |
400 |
275 |
85 |
730 |
635 |
128 |
1049 |
1012 |
The first column is the number of errors (ne = number of exceeded delays or number of wrong reports)
The second column is the number of samples for the pass limit (nsp , ns=Number of samples= number of successes + number of exceedings or number of reports)
The third column is the number of samples for the fail limit (nsf)
D.4.4 Pass fail decision rules for ECID, OTDOA, MBS, WLAN and BLE test cases
Having observed 0 errors, pass the test at 33+ samples, otherwise continue
Having observed 1 error, pass the test at 46+ samples, otherwise continue
Having observed 2 errors, pass the test at 58+ samples, fail the test at 2 samples, otherwise continue
Having observed 146 errors, pass the test at 1181+ samples, fail the test at 1173- samples, otherwise continue
Having observed 147 errors, fail the test at 1182- samples,
Where x+ means: x or more, x- means x or less
NOTE: an ideal DUT passes after 33 samples. The maximum test time is 1181 samples.
D.4.5 Background information to the pass fail limits
There is freedom to design the decision co-ordinates (ne, ns).
The binomial distribution and its inverse is used to design the pass and fail limits. Note that this method is not unique and that other methods exist.
Where
fail(..) is the error ratio for the fail limit
pass(..) is the error ratio for the pass limit
ER is the specified error ratio e.g. 0.05
ne is the number of bad results. This is the variable in both equations
M is the Bad DUT factor M=1.5
df is the wrong decision probability of a single (ne, ns) co-ordinate for the fail limit.
It is found by simulation to be df = 0.004
clp is the confidence level of a single (ne, ns) co-ordinate for the pass limit.
It is found by simulation to be clp = 0.9975
qnbinom(..): The inverse cumulative function of the negative binomial distribution
The simulation works as follows:
A large population of limit DUTs with true ER = 0.05 is decided against the pass and fail limits.
clp and df are tuned such that CL (95%) of the population passes and D (5%) of the population fails.
A population of Bad DUTs with true ER = M*0.05 is decided against the same pass and fail limits.
clp and df are tuned such that CL (95%) of the population fails and D (5%) of the population passes.
This procedure and the relationship to the measurement is justified in clause D.3.7. The number of DUTs decreases during the simulation, as the decided DUTs leave the population. That number decreases with an approximately exponential characteristics. After 169 bad results all DUTs of the population are decided.
NOTE: The exponential decrease of the population is an optimal design goal for the decision co-ordinates (ne, ns), which can be achieved with other formulas or methods as well.
Annex E (normative):
Conditions for ECID and OTDOA requirements