E.3 3GPP ATS implementation guidelines

34.123-33GPPPart 3: Abstract test suite (ATS)TSUser Equipment (UE) conformance specification

This clause provides a set of guidelines that must be followed during ATS development. In general, these guidelines are intended to prevent developers from making common errors, or discuss considerations that must be taken into account before using specific features of the TTCN language.

E.3.1 Test case groups shall reflect the TSS&TP document

Test groups shall be used to organize the test cases in the same way as the test purposes are structured in the prose specification.

The general structure of the test groups should be in the following format.

<protocol>/<group>/<subgroup>

E.g. RLC/UM/Segmentation/LengthIndicator7bit/

E.3.2 Test case names correspond to the clause number in the prose

Test case names are derived directly from the clause number in the prose specification. Decimal points between digits in the clause number are replaced with underscores. E.g. the test case name for the test purpose specified in clause 7.2.3.2 of 3GPP TS 34.123-1 [1] is tc_7_2_3_2. If more than one test case is required to achieve a test purpose, an additional digit may be added. See also ETR 141 [37], clause 6.3.7.

E.3.3 Use standard template for test case and test step header

Table E.3.3.1 illustrates how the Test Case dynamic behaviour header fields should be used.

Table E.3.3.1: Template for TTCN test case table header

Field			Contents
Test Case Name:			tc_NUMBER_OF_TESTCASE The number of the test case, which is used in the name of the test case, is the number it has in the prose specification. e.g.: "tc_26_13_1_3_1"
Group:			Is automatically filled and cannot be changed
Purpose:			This is taken directly from the prose specifications.
Configuration:			As required if concurrent TTCN is being used.
Default			The appropriate default
Comments:			First line contains: Specification: The names and clauses of relevant core specifications. Next line contains: Status: OK / NOT OK (+explanation if not ok) / Version number / Validated / Reviewed, etc. E.g.: Status: OK Rest of lines give comments as: What has to be done before running this test? E.g.: 1. Generic setup procedure must be completed before running this test. Any special information about what might be needed for the testing system, like specific requirements for the testing system, specific hacks, certain settings, etc. This field should be short (if long description is needed it must be put into Detailed Comments)
Selection Ref:			The appropriate test case selection expression.
Description:			Optional. Max 4 lines. If available, this should be the title of the prose clause. Note 1
Nr	Label	Behaviour Description		Constraints Ref	Verdict	Comments
1		Note 3		Note 3		Note 2
Detailed Comments			Contains detailed information about test steps + additional information Note 2
NOTE 1: The description field in the test case / step header is used to generate the test suite overview, and should only include a brief overview of the test case / step with a maximum of 4 lines. For a more detailed description of the test case / step algorithm / parameters etc, the comments or detailed comments fields should be used. NOTE 2: The comments field for each behaviour line should usually consist of a number that is a reference to a specific numbered comment in the detailed comments field. If this extra level of indirection reduces readability, brief comments can be used in the comments field for each behaviour line. NOTE 3: If entries in the behaviour description or constraints reference column contain lists with more than one element, carriage returns should be used between list elements to prevent the line from becoming too long.

Table E.3.3.2 illustrates how the Test Case dynamic behaviour header fields should be used.

Table E.3.3.2: Template for TTCN test step table header

Test Step Name			ts_TestStepName( p_Param1: Param1Type; p_Param2: Param2Type )
Group			Is automatically filled and cannot be changed
Objective			The objective of the test case. Provides a brief summary of the functionality of the test step.
Default			The appropriate default
Comments			A detailed description of the test step, including the relevant items from the following categories: Algorithm A detailed description of the algorithm / principles used within the test step Parameters: A description of each of the parameters passed to the test step, including the purpose of the parameter, valid values, restrictions etc. Preconditions The required state of the UE and / or SS before using this test step, including test steps that should be executed before using the present test step, and a description of all test case variables that must contain appropriate values before using this test step. Postcondidions The expected state of the UE and / or SS after using this test step, including a description of all test case variables that will be modified by this test step. NOTE: It is too difficult to maintain the list of variables required / affected by nested test steps, so it is the users responsibility to check which variables are required / affected by nested test steps.
Description			Optional. Max 4 lines. Note 1
Nr	Label	Behaviour Description				Constraints Ref	Verdict	Comments
1		Note 3				Note 3		Note 2
Detailed Comments			Contains detailed information about test steps + additional information Note 2
NOTE 1: The description field in the test case / step header is used to generate the test suite overview, and should only include a brief overview of the test case / step with a maximum of 4 lines. For a more detailed description of the test case / step algorithm / parameters etc, the comments or detailed comments fields should be used. NOTE 2: The comments field for each behaviour line should usually consist of a number that is a reference to a specific numbered comment in the detailed comments field. If this extra level of indirection reduces readability, brief comments can be used in the comments field for each behaviour line. NOTE 3: If entries in the behaviour description or constraints reference column contain lists with more than one element, carriage returns should be used between list elements to prevent the line from becoming too long.

E.3.4 Do not use identical tags in nested CHOICE constructions

A nested CHOICE requires tags in the different alternative type lists to differ (see ISO/IEC 8824 [29], clause 24.4, example 3, INCORRECT). "The tag shall be considered to be variable, … becomes equal to the tag of the "Type" … from which the value was taken".

EXAMPLE: components are defined in a nested CHOICE construction, but no distinguishing tags are used to make the difference between component types, i.e. tags for different types turn out to be identical.

Component ::= CHOICE {

gSMLocationRegistration_Components GSMLocationRegistration_Components,
gSMLocationCancellation_Components GSMLoactionCancellation_Components,
…
}

GSMLocationRegistration_Components ::= CHOICE {

gSMLocationRegistration_InvokeCpt [1] IMPLICIT GSMLocationRegistration_InvokeCpt,
gSMLocationRegistration_RRCpt [2] IMPLICIT GSMLocationRegistration_RRCpt,
gSMLocationRegistration_RECpt [3] IMPLICIT GSMLocationRegistration_RECpt,
gSMLocationRegistration_RejectCpt [4] IMPLICIT RejectComponent
}

GSMLocationCancellation_Components ::= CHOICE {

gSMLocationCancellation_InvokeCpt [1] IMPLICIT GSMLocationCancellation_InvokeCpt,
gSMLocationCancellation_RejectCpt [4] IMPLICIT RejectComponent
}

gSMLocationRegistrationInvokeCpt and gSMLocationCancellation_InvokeCpt have the same tag and can therefore not distinguished anymore. Note that ITEX 3.5 does not report this error.

E.3.5 Incorrect usage of enumerations

Enumerations may contain distinct integers only (see ISO/IEC 8824 [29], clause 15.1).

EXAMPLE: TypeOfNumber containing a NamedValueList in which there are non-distinct values.

TypeOfNumber ::= ENUMERATED {
…..,
internationalnumber (1),
level2RegionalNumber (1),
nationalNumber (2),
level1RegionalNumber (2),
……
}

E.3.6 Structured type as OCTETSTRING should not be used

"It is required to declare all fields of the PDUs that are defined in the relevant protocol standard, …" TR 101 101 [38] TTCN specification clause 11.15.1.

EXAMPLE 1: The ISDN Bearer Capability Information Element (BCAP) contents is defined as OCTETSTRING.

EXAMPLE 2: Usage of data type BITSTRING [7..15] as data type of the Call Reference (= 7 bits or =15 bits, but not 8 bits for example) does not correspond to the specification !!).

E.3.7 Wildcards in PDU constraints for structured types should not be used

Contrary to popular belief, TR 101 666 [27] does not support the use of wildcards for TTCN ASP parameters, or TTCN PDU fields whose type is structured. It is not clearly stated if wildcards are permitted for TTCN structured type elements whose type is structured but it is assumed that they are not permitted because the semantics for this are not clearly specified.

Note that this does not apply to ASN.1 Type definitions, ASPs, or PDUs.

Most tools do support wildcards for TTCN ASP parameters / TTCN PDU fields / TTCN structured type elements whose type is structured, but there is ambiguity between implementations since the semantics are not clearly specified in the core specification.

This feature is commonly used by TTCN developers, and is present in many existing test suites, including the 3GPP test suite, and in constraints that are being re-used from GERAN tests.

One problem with values ‘?’ and ‘*’ in constraints where they are used to indicate values of structured types, is that they would allow any combinations of values – even incorrect ones – which is not admissible according to the specifications. It is to be kept in mind that in tabular form each field is optional! It would be better to create and use an "any"-constraint which would deal with all the fields in detail (mandatory, IF PRESENT, etc.).

For the purpose of the present annex, the following rules shall apply:

1. ‘?’ shall not be used to indicate values of TTCN ASP parameters / TTCN PDU fields / TTCN structured type elements whose type is structured. Known TTCN implementations differ significantly in their implementation of this feature.

2. ‘*’ shall not be used for TTCN PDU fields, or TTCN ASP parameters whose type is structured (i.e. at the top level).

2.1 Usage of wildcards should be avoided in structured type identifiers. Only simple type fields should use * or ?

3. ‘*’ is permitted but discouraged for structured type elements whose type is structured. Note that this may result in ambiguous behaviour between TTCN implementations because the semantics are not specified in TR 101 666 [27].

4. One of the following two options shall be used as an alternative to using a ‘?’ for a TTCN ASP parameter / TTCN PDU field / TTCN structured type element whose type is structured.

4.1 Option 1: Use ‘*’ instead (only applicable to structured type elements due to rules 2 and 3 above).

WARNING: This may result in the situation where a UE omits a mandatory field, but passes the test anyway, and / or different behaviour depending on the TTCN tool used.

4.2 Option 2 (preferred option; supported by TR 101 666 [27]): Use an ‘any’ constraint, in conjunction with IF PRESENT if appropriate (whole TTCN ASP parameters / TTCN PDU fields / TTCN structured type elements may be omitted according to TR 101 666 [27]). This means that the constraint value specified for the parameter / field / element shall be a reference to another constraint of the appropriate structured type, which may in turn use wildcards for each of it’s elements according to the rules specified in the present annex.

5. A structured type formal parameter should not be used together with the IF_PRESENT indication inside a structured type constraint. If this is required, then this shall be clearly commented.

E.3.8 TSOs should be passed as many parameters as meaningful to facilitate their implementation

Parameters should be passed to TSOs to facilitate the TSO realization. If a TSO is used in various contexts, this should be reflected in the parameters passed to the TSO. Specifically, TSOs operating on well-defined (parameterized) constraints should take these constraints (including relevant parameters) as parameters if required.

BAD EXAMPLE: In this example, the TSO may be used in many contexts, but no information is passed to the TSO, which makes TSO realization difficult.

L?SETUPr (… tcv_invokeId := TSO_GET_INVOKEID ( ), …)

Sr (SU_GR3( GSM_IncomingCallMMInfo_Invoke(…)))

GOOD EXAMPLE: In this case, the TSO is provided with information about the data object from which the invoke Id is to be extracted, and the type of component from which the invoke Id is to be extracted is identified by passing the component constraint.

L?SETUPr (… tcv_invokeId := TSO_GET_INVOKEID ( DL_DataInd_Setup.msg, GSM_IncomingCallMMInfo_Invoke(…)), …)

Sr (SU_GR3( GSM_IncomingCallMMInfo_Invoke(…)))

To calculate the invocation identification and store the result in variable tcv_invokeId the TSO has to be provided with information about the data object from which the invoke Id is to be extracted. PDU constraint SU_GR3 may contain several components. In the specific situation only one of these components is relevant.

Depending on the nature of the TSO, passing the received value, or a subcomponent of the received value may be more appropriate than passing the constraint.

E.3.9 Specification of Encoding rules and variation should be indicated

TTCN does not mandate encoding rules, although TTCN foresees that applicable encoding rules and encoding variations can be indicated for the data structures used in a test suite.

There are standards defining encoding rules, e.g. the ITU-T Recommendation X.680 [39] series. However, the type of encoding called "Direct Encoding" – a bit-by-bit-mapping from the data definitions onto the data stream to be transmitted – is not defined anywhere. It therefore needs a "home".

TTCN should therefore define which encoding rules may legally be used by TTCN test suite specifiers. All the encoding rules defined in the ITU-T Recommendation X.680 [39] series should be contained in this repertoire. Additionally an encoding rule called Direct Encoding is needed in particular for tabular TTCN.

ITU-T Recommendation X.680 [39] allows to encode data objects using different length forms (short, long, indefinite). These could be used alternatively as encoding variations. Another encoding variation could be the "minimum encoding", accepting any of the length forms in reception, and using the shortest of the available forms in sending. The variation actually used has to be described somewhere (in the ATS).

E.3.10 Use of global data should be limited

The Phase 2 ATS became extremely complex due to the global definition of data. Data should be defined locally where possible if the language allows, alternatively the names of global constraints could be given prefixes to indicate their use.

E.3.11 Limit ATS scope to a single layer / sub-layer

Separate ATSs should be produced to test each Layer and perhaps sub Layer. By doing this preambles and common areas particular to one sub Layer can be confined to one test suite and parallel development of test suites can be facilitated.

E.3.12 Place system information in specially designed data structures

System Information data could be stored in specially defined data structures, use of these structures to build PDUs may help to ensure that a consistent set of data is transmitted in all the channels in a cell.

E.3.13 Place channel configuration in specially designed data structures

Likewise the configuration of a ‘channel’ could be stored in similar structures. This data can then be used to configure the test system and to build Assignment messages to the UE under test. This may help avoid the situation where the TTCN creates one channel and unintentionally commands the mobile to a different, non-existent, channel.

E.3.14 PICS / PIXIT parameters

It is desirable to limit the scope of PICS / PIXIT parameters.

A default value shall be provided in the PIXIT document for all PIXIT parameters.

PICS / PIXIT parameters shall not include structured types. If a structured parameter is required, several parameters shall be used, one for each simple element within the type, and a constraint shall be created to combine the simple parameters into a structured type.

For example, to use the following structured type as a parameter.

Type Name	LocAreaId_v
Encoding Variation
Comments	Location Area Identification Value 3GPP TS 24.008 [9] clause 10.5.1.3
Element Name	Type Definition	Field Encoding	Comments
mcc	HEXSTRING[3]		MCC 3 digits
mnc	HEXSTRING[3]		MNC 3 digits
lac	OCTETSTRING[2]		LAC
Detailed Comments

The following three PIXIT parameters should be defined: Parameter Name	Type	PICS/PIXIT Ref	Comments
px_LACDef	OCTETSTRING	PIXIT TC	default LAC
px_MCCDef	HEXSTRING	PIXIT TC	default MCC
px_MNCDef	HEXSTRING	PIXIT TC	default MNC

And then the following constraint can be used to combine the simple parameters into a structured parameter.

Constraint Name	cb_LocAreaIdDef_v
Structured Type	LocAreaId_v
Derivation Path
Encoding Variation
Comments
Element Name	Element Value	Element Encoding	Comments
mcc	px_MCCDef
mnc	px_MNCDef
lac	px_LACDef
Detailed Comments

E.3.15 Dynamic vs. static choices

Don’t use wildcards for static choice constraints. For example, a type that is similar for FDD and TDD should have 2 type definitions, rather than a single type that uses an ASN.1 choice. Then in the TTCN, the correct type should be selected based on test suite parameters.

E.g.:

[ pxUseTddMode ] AM ! TddSpecificAsp

AM ?

…

[ pxUseFddMode ] AM ! FddSpecificAsp

AM ? …

…

E.3.16 Definition of Pre-Ambles and Post Ambles

Test cases should, as far as possible, use one of a set of standard pre-ambles to place the user equipment in its initial conditions. These pre-ambles should align with the generic setup procedures in the conformance specification. All non-standard pre-ambles should be identified and added to the pre-amble library.

With pre-ambles readability is very important so they should not use other test steps to send message sequences, and they should be passed as few parameters as possible. This also makes the results log easier to read.

The prose message sequence charts should be analysed, and a catalogue of common ways in which the test cases can terminate (correctly or incorrectly) created. This catalogue should be used to create a set of post-ambles. All final verdicts should be assigned in the post-ambles.

Wherever possible, a post-amble should return the test system and the User Equipment under test to a known idle state.

E.3.17 Use test steps to encapsulate AT and MMI commands

When the same AT or MMI command is to be used more than once within a test suite, the command should be placed within a test step, to ensure that the same information is provided consistently. The main intention of this guideline is to ensure that MMI commands provided to the user are consistent, and can be changed easily if required.

For example, a test step similar to the one illustrated in table E.3.17 should be created and attached so that the same information is provided to the user each time the test step is used, and the string to be sent only exists in one place within the test suite.

Table E.3.17: Example test step to encapsulate AT / MMI commandsDefault behaviour

Test Step Name			ts_AT_MMI_Example
Group
Objective			Send an MMI command instructing the user to insert the USIM card into the UE.
Default
Comments			Encapsulate an AT / MMI command within a test step to ensure that the same information is used consistently, and the information only exists in one place within the test suite.
Description
Nr	Label	Behaviour Description		Constraints Ref	Verdict	Comments
1		Ut ! MMI_CmdReq		ca_MMICmdReq ( " Please insert the USIM card into the UE ")
2		Ut ? MMI_CmdCnf		ca_MMICmdCnf

Defaults are test steps that are executed when ever a receive event occurs that is not expected. Not expected means that it does not match any of the defined ASP constraints at that point in the test case. The default behaviour used in test case is defined in the test case declaration. They can be defined to stop the test case by calling a standard post-amble or receive the event as OTHERWISE and RETURN back to step where the unexpected event occurred.

A strategy for dealing with unexpected behaviour involving consistent use of defaults should be developed, and applied to test cases wherever possible.

If during a test case or test step it is necessary to change the default behaviour, the ACTIVATE statement may be used.

E.3.18 Use system failure guard timers

A timer should be set at the beginning of each test case to guard against system failure. Behaviour on expiry of this timer should be consistent for all test cases.

E.3.19 Mapping between prose specification and individual test cases

The ATS should map one-to-one between test cases and tests as described in 3GPP TS 34.123-1 [1]. A method for ensuring that the two specifications track each other needs to be defined.

E.3.20 Verdict assignment

E.3.20.1 General

Final verdicts shall only be used to indicate test case errors, or when unexpected UE behaviour occurs such that it not sensible to continue the test. When a test case reaches a leaf node, the test case ends, and the current preliminary verdict is assigned. At least one preliminary verdict shall be assigned for every test case. If a test case terminates and no final or preliminary verdicts have been assigned, the current value of the predefined variable R will be ‘none’, and a test case error is recorded instead of a final verdict.

Labels shall be used for every line in which a verdict is posted to improve the traceability of the conformance log produced when the test case is executed. These labels should be kept short, since they appear in the dynamic behaviour tables.

All test suites shall make use of a global boolean variable, defined in the common module, called tcv_TestBody. tcv_TestBody is updated within each test case to indicate if the test body is currently being executed. tcv_TestBody is referenced in defaults and test steps to assign a preliminary inconclusive verdict when unexpected events occur outside of the test body, or a preliminary failure verdict when unexpected events occur within the test body.

The initial value in the declaration of the test case variable tcv_TestBody shall be FALSE. The variable will be bound to this value when the ATS is initialized, and will be re-bound to this value after termination of each test case, ready for execution of the next test case.

E.3.20.2 Test cases

A line similar to line 3 in table E.3.20.2 shall be used in all test cases to set tcv_TestBody to TRUE. This line shall have the label TBS to indicate the Test Body Start point.

A line similar to line 6 in table E.3.20.2 shall be used in all test cases to set tcv_TestBody to FALSE. This line shall have the label TBE[N] to indicate the Test Body End point. A number N (with one or more digits) may optionally be appended to the label to distinguish between multiple test body end points. If the number of possible test sequences makes management of the tcv_TestBody variable too difficult, the variable can be set to TRUE at the beginning of the test. In this case, a comment shall be added to the test case noting that tcv_TestBody is not updated, so verdicts assigned within preambles and postambles will be treated as if they are part of the test body.

Within the test body, preliminary verdicts shall be used to indicate the result of the test purpose. Each behaviour line within the test body containing a preliminary verdict shall have a label of the form TBXN, where X is one of P, F, I for pass, fail, and inconclusive respectively, and N is a number (with one or more digits) used to distinguish multiple TBPs, TBFs, or TBIs in the same test case.

If an unexpected event occurs corresponding to a test case error, a final inconclusive verdict shall be assigned, and the behaviour line shall have a label ERRN, where N is a number used to distinguish multiple ERRs, and ERR indicates that a test case error has occurred. An example of this is provided in the test step clause.

Table E.3.20.2 contains an example test case illustrating these concepts.

In case of a failure event of a time consumed test case (longer than 30 minutes), the test case can be stopped by using a final verdict after the execution of the postamble.

Table E.3.20.2: Example test case illustrating use of verdicts, labels
and tcv_TestBody test case variable

Nr	Label	Behaviour Description		Constraints Ref	Verdict	Comments
1		+ts_Preambles
2	TBS	( tcv_TestBody := TRUE )				1
3		L ! Stimulus		cs_Stimulus1
4		+lt_Response
5	TBE	(tcv_TestBody := FALSE )			(P)	2
6		+ts_Postambles
7		lt_Response
8	TBP1	L ? Response		cr_ValidResponse1	(P)	3
9	TBP2	L ? Response		cr_ValidResponse2	(P)	3
10	TBF1	L ? Response		cr_InvalidResponse	(F)	4
11	TBI1	L ? Response		cr_OtherResponse	(I)	5
Detailed comments			The behaviour line setting tcv_TestBody to TRUE shall have the label TBS. The behaviour line setting tcv_TestBody to FALSE shall have the label TBE, and can optionally be used to assign a verdict indicating that the test purpose has passed or failed (i.e. if the final behaviour statement in the test body is a tree attachment). The label TBPN is used to indicate that the test purpose has been achieved via the Nth possible valid UE behaviour. The label TBFN is used to indicate that the test purpose has not been achieved, due to the Nth possible failure cause. The label TBIN is used to indicate that the test result is inconclusive for the Nth possible unexpected / unknown event.

E.3.20.3 Test steps

To promote re-use, test steps shall only assign preliminary verdicts (I) and (F). (P) verdicts shall be managed at the test case level in general, but may be used sparingly within test steps. ETR 141 [37] clause 12.4 recommends that a preliminary pass verdict should be assigned at the leaf of each passing event sequence of the test step. If a test step includes an alternative for unexpected / invalid behaviour, then either a preliminary inconclusive verdict shall be assigned if tcv_TestBody is FALSE, or a preliminary failure verdict shall be assigned if tcv_TestBody is TRUE.

Each behaviour line within the test step containing a preliminary verdict shall have a label of the form TSXN, where X is one of P, F or I for pass, fail, and inconclusive respectively, and N is a number (with one or more digits) used to distinguish multiple TSPs, TSFs, or TSIs in the same test step.

If an unexpected event occurs corresponding to a test case error, a final inconclusive verdict shall be assigned, and the behaviour line shall have a label ERRN, where N is a number used to distinguish multiple ERRs, and ERR indicates that a test case error has occurred.

Table E.3.20.3 contains an example test step illustrating these concepts.

Table E.3.20.3: Example test step illustrating use of verdicts, labels
and tcv_TestBody test case variable

Nr	Label	Behaviour Description		Constraints Ref	Verdict	Comments
1		[ p_Mode = tsc_Mode1 ]
2		L ! Stimulus		cs_Stimulus1
3		+lt_Response
4		[ p_Mode = tsc_Mode2 ]
5		L ! Stimulus		cs_Stimulus2
6		+lt_Response
7	ERR1	[ TRUE ]			I	1
8		lt_Response
9		L ? Response		cr_ValidResponse1		2
10		L ? Response		cr_InvalidResponse
11	TSI1	[ tcv_TestBody = FALSE ]			(I)	3
12	TSF1	[ tcv_TestBody = TRUE ]			(F)	4
Detailed comments			An invalid value for the parameter p_Mode has been passed to this test step, so a final inconclusive verdict is assigned, with a label indicating that a test case error has occurred. If the expected behaviour occurs, then the test step completes at the leaf node, and the current preliminary verdict is not changed. If unexpected / invalid behaviour occurs, and the current test step is being used as a preamble or postamble ( tcv_TestBody = FALSE ) then a preliminary inconclusive verdict is assigned. If unexpected / invalid behaviour occurs, and the current test step is being used as part of the test purpose( tcv_TestBody = TRUE ) then a preliminary failure verdict is assigned.

E.3.20.4 Defaults

Each behaviour line within a default behaviour table containing a preliminary verdict shall have a label of the form DFXN, where X is one of F or I for fail, and inconclusive respectively, and N is a number (with one or more digits) used to distinguish multiple DFFs, or DFIs in the same test step.

tcv_TestBody shall be referenced from within default behaviour tables to assign the appropriate verdict when unexpected events occur.

Table E.3.20.4 contains an example default behaviour table illustrating these concepts.

Table E.3.20.4: Example default behaviour table illustrating use of verdicts,
labels and tcv_TestBody test case variable

Nr	Label	Behaviour Description		Constraints Ref	Verdict	Comments
1		L ? Response		cr_IgnoredResponse		1
2		RETURN
3	DFI1	L ? OTHERWISE [ tcv_TestBody = FALSE ]			(I)	2
4	DFF1	L ? OTHERWISE [ tcv_TestBody = TRUE ]			(F)	3
Detailed comments			1. Valid events that are to be ignored can be included in the default behaviour, but should have no preliminary verdict assigned. 2. If unexpected data is received in the preambles or postambles, a preliminary inconclusive verdict is assigned, and the test case is terminated. 3. If unexpected data is received in the test body, a preliminary failure verdict is assigned, and the test case is terminated.

See also ETR 141 [37], clauses 11.2, 12.4 and 14.3.

E.3.21 Test suite and test case variables

A default value shall be provided for all test suite and test case variables.

E.3.22 Use of macros is forbidden

The use of macros is forbidden, to support migration to TTCN3.

E.3.23 Support for future Radio Access Technologies

To allow existing test cases to be updated in future to support other radio access technologies, test suites shall make use of a PIXIT parameter px_RAT of type RatType as shown in the following example.

Test Case Name

tc_RAT_Example1

Nr	Label	Behaviour Description	Constraints Ref	Verdict	Comments
1		START t_Guard( 300 )
2		[ px_RAT = fdd ]
3		PCO ! FDD_PDU	c_FDD_PDU1		FDD specific behaviour
4	TBP1	PCO ? COMMON_PDU	c_COMMON_PDU1	(P)
5		[ px_RAT = tdd ]
6		PCO ! TDD_PDU	c_TDD_PDU1		TDD specific behaviour
7	TBP2	PCO ? COMMON_PDU	c_COMMON_PDU1	(P)
8		[ px_RAT = other_rat ]		I	Tests for this RAT not implemented yet
9	TCE1	[ TRUE ]		I	Unexpected px_RAT value

Detailed Comments

In general, alternatives should be used to separate behaviour specific for each RAT, and common behaviour should be re-used as much as possible. A final inconclusive verdict shall be used for any alternatives that have not been implemented yet.

Local trees may be used as shown in the following example to improve re-use of common behaviour.

Test Case Name

tc_RAT_Example2

Nr	Label	Behaviour Description	Constraints Ref	Verdict	Comments
1		START t_Guard( 300 )
2		+lt_RAT_SpecificPart
3	TBP1	PCO ? COMMON_PDU	c_COMMON_PDU1	(P)	Common behaviour
		lt_RAT_SpecificPart
4		[ px_RAT = fdd ]
5		PCO ! FDD_PDU	c_FDD_PDU1		FDD specific behaviour
6		[ px_RAT = tdd ]
7		PCO ! TDD_PDU	c_TDD_PDU1		TDD specific behaviour
8	TCE1	[ TRUE ]		(I)	Unexpected px_RAT value

Detailed Comments

E.3.24 Managing multiple representations of the same information

When the same information is represented using multiple types within the same test suite, it is necessary to manage conversions between the types, and ensure that the information remains consistent across all of the representations.

For example, IMSI is represented as ‘SEQUENCE (SIZE (6..15)) OF Digit’ in the RRC ASN.1 definitions, as a HEXSTRING for input as a PIXIT parameter, and as an information element defined in TTCN tabular format for MM.

E.3.24.1 Predefined types

Conversion operations are not required to convert the following TTCN predefined types to their counterparts in ASN.1.

a) INTEGER predefined type.

b) BOOLEAN predefined type.

c) BITSTRING predefined type.

d) HEXSTRING predefined type.

e) OCTETSTRING predefined type.

f) OBJECTIDENTIFIER predefined type.

g) R_TYPE predefined type.

h) CharacterString predefined types.

Therefore it is valid to pass a value of type BIT STRING (ASN.1) as a formal parameter of type BITSTRING (TTCN predefined).

E.3.24.2 Simple types

TR 101 666 [27], clause 11.2.1 states:

– "TTCN is a weakly typed language, in that values of any two types which have the same base type are considered to be type compatible (e.g. for the purposes of performing assignments or parameter passing)".

When simple types have restrictions, it is the TTCN author’s responsibility to ensure that the restrictions are compatible. The TTCN compiler provides some assistance with this, but the extent of the checking is compiler specific.

E.3.24.3 Structured types

For conversion between more complex representations, test suite operations will generally be required. If the mapping is simple enough, it may be possible to perform the conversion using a test step, which takes the common representation as a parameter, and stores the required representation in a test case variable. This may avoid the need for an extra test suite operation.

In TTCN-2 the NAS UL PDU types are frozen from Rel-11 onward, a generic extension field is used to match any received extension sent by UE. For example, the field “extensionFields” in added as the last field of PAGINGRESPONSE.

PDU Name	PAGINGRESPONSE
TCO Type	Dc_SAP
Encoding Rule Name
Encoding Variation
Comments	PAGINGRESPONSE ue -> n 44.018 cl 9.1.25
Element Name	Type Definition	Field Encoding	Comments
skipIndicator	SkipIndicator		Skip Indicator M BITSTRING [4]
rRProtocolDiscriminator	ProtocolDiscriminator		RR Protocol Discriminator M BITSTRING [4]
msgType	MsgType		Message Type (1) M BITSTRING [8]
spare4	B4		Spare half octet M BITSTRING [4]
ciphKeySeqNum	CiphKeySeqNum		Ciphering Key Sequence Number M BITSTRING [4]
mSClsmk2	MS_Clsmk2_lv		Mobile Station Classmark 2 M MSClsmk2 (4 octets)
mobileId	MS_Identity_lv		Mobile Identity LV M MobileId (2-10 octets)
extensionFields	ExtensionFields_Type		Extension field used to receive any rel-11 or later IE added to UL PDU types
Detailed Comments

E.3.24.4 Conversion responsibility

Two design approaches are possible for deciding where the responsibility of conversion lies: Calling party conversion and called party conversion.

The appropriate option should be selected on a case-by-case basis with the following restrictions:

– If one representation of the information is a PIXIT parameter, and this information must be passed to a test step, the called party conversion option shall be used, and the formal parameter to the test step shall always have the same type as the PIXIT parameter.

– If a test step provides multiple alternatives for different radio access technologies, which require different representations of the same information, the called party conversion convention shall be used. In this case a technology independent representation of the information shall be passed as a parameter, and the test step shall perform the conversion to the appropriate type depending on which RAT is being used.

E.3.24.5 Option 1: Calling party conversions

For this approach, each test step provides an interface based on its internal representation. It is the responsibility of the test case / step attaching the test step to perform the conversion before the attachment.

E.3.24.5.1 Advantages

– The number of calls to conversion operations is minimized.

– The complexity of the attached test steps is reduced because fewer conversions are required than for the called party conversion approach.

E.3.24.5.2 Disadvantages

– Different types are used to transfer the same information across the test step interfaces.

– The complexity of the attaching test steps / cases may be increased because conversions are required before attaching a test step.

– The attaching test steps / cases are responsible for ensuring that multiple representations contain consistent information.

E.3.24.6 Option 2: Called party conversions

In this case, the same representation is used wherever the information must be used as a formal parameter value to a test step, and it is the responsibility of the test step to perform any conversions required.

E.3.24.6.1 Advantages

– The complexity in the attaching test case / step is reduced, which will often improve readability.

– The test step interfaces are cleaner, because the same representation is always passed as a formal parameter.

– Internal representations may be hidden within test steps so that calling parties do not need to have any knowledge of them.

E.3.24.6.2 Disadvantages

– Conversion operations may be called more times than necessary, for example if the same test step is attached twice within one test case.

E.3.25 Assignment using constraint

According to TR 101 666 [27], the Right Hand Side (RHS) of an assignment shall not contain any unbound variables. The matching symbols, AnyValue or AnyOrOmit, in both tabular and ASN.1 constraints shall not be assigned to a test case variable, independent of the type of the test case variable.

E.3.26 Guidelines for use of timers when tolerances are applicable

Timed events within the test suite should implement the timer tolerances specified in 3GPP TS 34.108 [3], clause 4.2.3. It is the TTCN author’s responsibility to ensure that appropriate tolerance checks and tolerance values are being used.

NOTE: Tolerances are not applicable to guard timers as described in clause E.3.18 of the present document.

E.3.26.1 Specific situations

The present clause provides recommendations for how to implement timers with tolerances for the following situations:

a) The timed event must occur before a given time.

b) The timed event must occur after a given time.

c) The timed event must occur between two given times.

NOTE: A specific case of this situation is when the desired event occurs at a specific time, plus or minus a tolerance.

E.3.26.2 Example situations

The examples below assume:

a) The test case variable tcv_Duration contains the timer duration (in terms of the units used in the timer declaration).

b) The test case variable tcv_Tolerance has been initialized using one of the following assignments (it is the TTCN author’s responsibility to select the calculation resulting in the greatest value of tcv_Tolerance. Reference 3GPP TS 34.108 [3], clause 4.2.3):

1) ( tcv_Tolerance := tcv_Duration / 10 )

2) ( tcv_Tolerance := 2 * tcv_TTI + tsc_T_Delta )
Where tcv_TTI contains the applicable TTI (in ms), and tsc_T_Delta is 55 ms.

NOTE: The timer value parameters used when starting the timers in the examples are recommendations only. Other timer value parameter expressions may be used if appropriate.

E.3.26.2.1 Example of situation 1

Test Step Name			ts_TimerSituation1Example
Purpose			To demonstrate implementation of a timed event that must occur before a given time.
Nr	Label	Behaviour Description			Constraints Ref	Verdict	Comments
1		START t_UpperBound ( tcv_Duration + tcv_Tolerance )					1.
2		+lt_TimedEvent					2.
3	TSP1	CANCEL t_UpperBound				(P)	3.
4	TSF1	? TIMEOUT t_UpperBound				(F)	4.
		lt_TimedEvent
5		[ TRUE ]					2.
Detailed Comments				1. Start the timer, allowing tcv_Tolerance extra units for the timed event to arrive. 2. The timed event is observed. 3. The timed event occurred before the timeout, so cancel the timer, and assign a preliminary pass verdict. 4. The timer expired before the timed event occurred, so assign a preliminary failure verdict.

E.3.26.2.2 Example of situation 2

Test Step Name			ts_TimerSituation2Example
Purpose			To demonstrate implementation of a timed event that must occur after a given time.
Nr	Label	Behaviour Description			Constraints Ref	Verdict	Comments
1		START t_LowerBound ( tcv_Duration – tcv_Tolerance )					1.
2		? TIMEOUT t_LowerBound					2.
3		+lt_TimedEvent					3.
4	TSP1	[ TRUE ]				(P)	3.
5		+lt_TimedEvent					4.
6	TSF1	CANCEL t_LowerBound				(F)	4.
		lt_TimedEvent
7		[ TRUE ]
Detailed Comments				1. Start the timer, allowing tcv_Tolerance extra units for the timed event to arrive. 2. The timeout is observed before the timed event. 3. The timed event is observed, so assign a preliminary pass verdict. 4. The timed event occurred before the timeout, so cancel the timer, and assign a preliminary failure verdict.

E.3.26.2.3 Example of situation 3

Test Step Name			ts_TimerSituation3Example
Purpose			To demonstrate implementation of a timed event that must occur between two given times.
Nr	Label	Behaviour Description			Constraints Ref	Verdict	Comments
1		START t_UpperBound ( tcv_Duration + tcv_Tolerance ), START t_LowerBound ( tcv_Duration – tcv_Tolerance )					1.
2		? TIMEOUT t_LowerBound					2.
3		+lt_TimedEvent					3.
4	TSP1	CANCEL t_UpperBound				(P)	3.
5	TSF1	? TIMEOUT t_UpperBound				(F)	4.
6		+lt_TimedEvent					5.
7	TSF2	CANCEL t_LowerBound , CANCEL t_UpperBound				(F)
		lt_TimedEvent
8		[ TRUE ]
Detailed Comments				1. Start the upper and lower bound timers, allowing tcv_Tolerance extra units each side of the expected time for the timed event to arrive. 2. The lower bound timeout is observed before the timed event. 3. The timed event is observed, so cancel the upper bound timer, and a preliminary pass verdict is assigned. 4. The upper bound timer expired before the timed event occurred, so a preliminary failure verdict is assigned. 5. The timed event occurred before the lower bound timer expired, so a preliminary failure verdict is assigned.

Annex F (informative):
Void

Void.

Annex G (informative):
Recommendation of an unique ICS/IXIT electronic exchange format

With standardization of ICS/IXIT file format, same Test Suite Parameter (TSP) files can be used across different System Simulators. The ICS/PIXIT will be simple ASCII text files. The assumption is that the test suite parameters are of simple type definitions only and do not include structured types (clause E.3.14).