D.4 Pass fail decisions

37.571-13GPPPart 1: Conformance test specificationRelease 16TSUser Equipment (UE) conformance specification for UE positioning

D.4.1 Numerical definition of the pass fail limits for A-GNSS test cases

Table D.4.1: Pass/fail limits for A-GNSS test cases

ne

nsp

nsf

ne

nsp

nsf

ne

nsp

nsf

ne

nsp

nsf

0

77

NA

43

855

576

86

1525

1297

129

2173

2050

1

106

NA

44

871

592

87

1540

1314

130

2188

2067

2

131

NA

45

887

608

88

1556

1331

131

2203

2085

3

154

NA

46

903

625

89

1571

1349

132

2218

2103

4

176

NA

47

919

641

90

1586

1366

133

2233

2121

5

197

NA

48

935

657

91

1601

1383

134

2248

2139

6

218

42

49

951

674

92

1617

1401

135

2263

2156

7

238

52

50

967

690

93

1632

1418

136

2277

2174

8

257

64

51

982

706

94

1647

1435

137

2292

2192

9

277

75

52

998

723

95

1662

1453

138

2307

2210

10

295

87

53

1014

739

96

1677

1470

139

2322

2227

11

314

100

54

1030

756

97

1692

1487

140

2337

2245

12

333

112

55

1046

772

98

1708

1505

141

2352

2263

13

351

125

56

1061

789

99

1723

1522

142

2367

2281

14

369

139

57

1077

805

100

1738

1540

143

2381

2299

15

387

152

58

1093

822

101

1753

1557

144

2396

2317

16

405

166

59

1108

839

102

1768

1574

145

2411

2335

17

422

180

60

1124

855

103

1783

1592

146

2426

2352

18

440

194

61

1140

872

104

1798

1609

147

2441

2370

19

457

208

62

1155

889

105

1813

1627

148

2456

2388

20

474

222

63

1171

906

106

1828

1644

149

2470

2406

21

492

237

64

1186

922

107

1844

1662

150

2485

2424

22

509

251

65

1202

939

108

1859

1679

151

2500

2442

23

526

266

66

1217

956

109

1874

1697

152

2515

2460

24

543

281

67

1233

973

110

1889

1714

153

2530

2478

25

560

295

68

1248

990

111

1904

1732

154

2544

2496

26

577

310

69

1264

1007

112

1919

1750

155

2559

2513

27

593

325

70

1279

1024

113

1934

1767

156

2574

2531

28

610

341

71

1295

1040

114

1949

1785

157

2589

2549

29

627

356

72

1310

1057

115

1964

1802

158

2603

2567

30

643

371

73

1326

1074

116

1979

1820

159

2618

2585

31

660

387

74

1341

1091

117

1994

1838

160

2633

2603

32

676

402

75

1357

1108

118

2009

1855

161

2648

2621

33

693

418

76

1372

1126

119

2024

1873

162

2662

2639

34

709

433

77

1387

1143

120

2039

1890

163

2677

2657

35

725

449

78

1403

1160

121

2054

1908

164

2692

2675

36

742

465

79

1418

1177

122

2069

1926

165

2707

2693

37

758

480

80

1433

1194

123

2084

1943

166

2721

2711

38

774

496

81

1449

1211

124

2099

1961

167

2736

2729

39

790

512

82

1464

1228

125

2114

1979

168

2751

2747

40

807

528

83

1479

1245

126

2128

1997

169

2765

NA

41

823

544

84

1495

1263

127

2143

2014

42

839

560

85

1510

1280

128

2158

2032

NOTE: The first column is the number of bad results (ne)
The second column is the number of results for the pass limit (nsp)
The third column is the number of results for the fail limit (nsf)

D.4.2 Pass fail decision rules for A-GNSS test cases

Having observed 0 bad results, pass the test at ≥77 results, otherwise continue

Having observed 1 bad result, pass the test at ≥106 results, otherwise continue

Having observed 2 bad results, pass the test at ≥131 results, otherwise continue

etc. until

Having observed 6 bad results, pass the test at ≥218 results, fail the test at ≤ 42 results, otherwise continue

Having observed 7 bad results, pass the test at ≥238 results, fail the test at ≤ 52 results, otherwise continue

etc. until

Having observed 168 bad results, pass the test at ≥2751 results, fail the test at ≤2747 results, otherwise continue

Having observed 169 bad results, pass the test at ≥2765 results, otherwise fail

NOTE: an ideal DUT passes after 77 results. The maximum test time is 2765 results.

D.4.3 Numerical definition of the pass fail limits for ECID, OTDOA, MBS, WLAN and BLE test cases

Table D.4.3: Pass/fail limits for ECID, OTDOA, MBS, WLAN and BLE test cases

ne

nsp

nsf

ne

nsp

nsf

ne

nsp

nsf

ne

nsp

nsf

0

33

NA

43

408

283

86

737

644

129

1056

1021

1

46

NA

44

416

291

87

745

653

130

1064

1030

2

58

2

45

424

299

88

752

661

131

1071

1039

3

69

5

46

432

307

89

760

670

132

1078

1048

4

79

8

47

440

315

90

767

679

133

1086

1057

5

89

12

48

447

324

91

775

687

134

1093

1066

6

99

17

49

455

332

92

782

696

135

1100

1074

7

109

22

50

463

340

93

790

705

136

1108

1083

8

118

27

51

471

348

94

797

713

137

1115

1092

9

127

33

52

478

356

95

804

722

138

1122

1101

10

136

39

53

486

365

96

812

731

139

1130

1110

11

145

45

54

494

373

97

819

739

140

1137

1119

12

154

51

55

502

381

98

827

748

141

1144

1128

13

163

58

56

509

389

99

834

757

142

1152

1137

14

172

64

57

517

398

100

842

766

143

1159

1147

15

180

71

58

525

406

101

849

774

144

1166

1155

16

189

78

59

532

414

102

857

783

145

1174

1164

17

197

85

60

540

423

103

864

792

146

1181

1173

18

206

92

61

548

431

104

871

801

147

NA

1182

19

214

99

62

555

440

105

879

809

148

20

223

106

63

563

448

106

886

818

149

21

231

113

64

571

456

107

894

827

150

22

239

120

65

578

465

108

901

836

151

23

248

128

66

586

473

109

909

844

152

24

256

135

67

594

482

110

916

853

153

25

264

142

68

601

490

111

923

862

154

26

272

150

69

609

499

112

931

871

155

27

281

157

70

616

507

113

938

880

156

28

289

165

71

624

516

114

946

888

157

29

297

173

72

632

524

115

953

897

158

30

305

180

73

639

533

116

960

906

159

31

313

188

74

647

541

117

968

915

160

32

321

196

75

654

550

118

975

924

161

33

329

204

76

662

558

119

983

933

162

34

337

211

77

669

567

120

990

941

163

35

345

219

78

677

575

121

997

950

164

36

353

227

79

684

584

122

1005

959

165

37

361

235

80

692

592

123

1012

968

166

38

369

243

81

700

601

124

1019

977

167

39

377

251

82

707

610

125

1027

986

168

40

385

259

83

715

618

126

1034

994

169

41

393

267

84

722

627

127

1042

1003

42

400

275

85

730

635

128

1049

1012

The first column is the number of errors (ne = number of exceeded delays or number of wrong reports)

The second column is the number of samples for the pass limit (nsp , ns=Number of samples= number of successes + number of exceedings or number of reports)

The third column is the number of samples for the fail limit (nsf)

D.4.4 Pass fail decision rules for ECID, OTDOA, MBS, WLAN and BLE test cases

Having observed 0 errors, pass the test at 33+ samples, otherwise continue

Having observed 1 error, pass the test at 46+ samples, otherwise continue

Having observed 2 errors, pass the test at 58+ samples, fail the test at 2 samples, otherwise continue

Having observed 146 errors, pass the test at 1181+ samples, fail the test at 1173- samples, otherwise continue

Having observed 147 errors, fail the test at 1182- samples,

Where x+ means: x or more, x- means x or less

NOTE: an ideal DUT passes after 33 samples. The maximum test time is 1181 samples.

D.4.5 Background information to the pass fail limits

There is freedom to design the decision co-ordinates (ne, ns).

The binomial distribution and its inverse is used to design the pass and fail limits. Note that this method is not unique and that other methods exist.

Where

fail(..) is the error ratio for the fail limit

pass(..) is the error ratio for the pass limit

ER is the specified error ratio e.g. 0.05

ne is the number of bad results. This is the variable in both equations

M is the Bad DUT factor M=1.5

df is the wrong decision probability of a single (ne, ns) co-ordinate for the fail limit.
It is found by simulation to be df = 0.004

clp is the confidence level of a single (ne, ns) co-ordinate for the pass limit.
It is found by simulation to be clp = 0.9975

qnbinom(..): The inverse cumulative function of the negative binomial distribution

The simulation works as follows:

A large population of limit DUTs with true ER = 0.05 is decided against the pass and fail limits.

clp and df are tuned such that CL (95%) of the population passes and D (5%) of the population fails.

A population of Bad DUTs with true ER = M*0.05 is decided against the same pass and fail limits.

clp and df are tuned such that CL (95%) of the population fails and D (5%) of the population passes.

This procedure and the relationship to the measurement is justified in clause D.3.7. The number of DUTs decreases during the simulation, as the decided DUTs leave the population. That number decreases with an approximately exponential characteristics. After 169 bad results all DUTs of the population are decided.

NOTE: The exponential decrease of the population is an optimal design goal for the decision co-ordinates (ne, ns), which can be achieved with other formulas or methods as well.

Annex E (normative):
Conditions for ECID and OTDOA requirements