6 Encoder conformance
26.4063GPPConformance testingEnhanced aacPlus general audio codecGeneral audio codec audio processing functionsRelease 17TS
6.1 Floating point encoder
No specific routines for floating-point encoder conformance testing are defined. It is recommended to use the
floating-point code from 3GPP TS 26.410 [5]. In addition, it is recommended to verify that the implementation meets the criteria defined in clause 6.2 (Fixed-point encoder). If the floating-point code is used for an implementation in mobile equipment, the criteria defined in 6.2 shall be met.
6.2 Fixed point encoder
Conformance of fixed-point encoder implementations for use in mobile equipment shall be verified by bit-exact behaviour to the fixed-point reference code [6] as defined in 6.2.1, or by meeting the objective criteria as defined in 6.2.2, or by performing subjective tests as described in 6.2.3. Fixed-point encoder implementations which are not used in mobile equipment, should meet the conformance criteria defined in this clause.
Bit-exact behaviour should be preferred for fixed-point implementations where it can be achieved without undue penalty on computational complexity.
If an implementer chooses to implement only a mono-encoder functionality, then conformance of only this functionality shall be tested. This shall apply irrespective of the conformance testing method chosen.
6.2.1 Bit-exact Behaviour
If the bit-exact conformance method is chosen, the conformance of a fixed-point encoder implementation under test shall be verified by checking the bit-exactness of the raw payload of the encoded output as produced by the fixed-point reference encoder code [6] for all 23 test vectors as given in Annex B.
6.2.2 Objective criteria
If the objective criteria conformance method is chosen, the conformance of a fixed-point encoder implementation under test shall be verified by checking objective conformance criteria for each of the three tools of the Enhanced aacPlus encoder: AAC, SBR and PS. The tests shall be conducted with the test vectors as given in Annex B. The configurations under test shall be 16 kbit/s mono, 32 kbit/s parametric stereo and 48 kbit/s stereo. All tests shall use the fixed-point encoder as reference.
For the purpose of fixed-point encoder conformance testing a tool has been developed which can be used to extract the various parts of an Enhanced aacPlus bitstream such that conformance of each of those parts can be tested independently. The source code for this tool can be found in an electronic attachment (fixp_enc_test_tool.zip) to this document. For a description on how to invoke the various conformance tests, please refer to the "readme.txt" file that is part of the attached package. This tool is henceforth referenced as "fixed-point encoder test tool".
6.2.2.1 AAC encoder conformance
Conformance testing of the AAC encoder tool shall be conducted using an implementation of ITU-R BS.1387 (PEAQ) [9]. The test consists of a statistical analysis comparison of the reference encoder performance with the performance of the encoder under test. The following pseudo-code and the succeeding description outlines how the performance shall be assessed.
|
cfg = 0; |
|
foreach (bitrate, channels, aacbandwidth) { |
|
tv = 0; |
|
foreach (item) { |
|
bandlimit (item, item_bandlimited, aacbandwidth); |
|
encoder_reference (item, reference_bitstream, bitrate, channels); |
|
encoder_test (item, test_bitstream, bitrate, channels); |
|
decoder_bandlimiting (reference_bitstream, reference_waveform); |
|
decoder_bandlimiting (test_bitstream, test_waveform); |
|
odg_ difference [cfg][tv] = PEAQ_advanced (item_bandlimited, reference_waveform) – |
|
tv++; |
|
} |
|
cfg++; |
|
} |
where:
– bandlimit (in, out, bw) is a function that limits the bandwidth of the waveform ‘in’ to ‘bw’ and stores the result in the waveform ‘out’
– encoder_reference (in, out, br, ch) is a function that calls the reference encoder such that the waveform ‘in’ is encoded to the bitstream ‘out’ at the bitrate ‘br’ and using channelmode ‘ch’
– encoder_test (in, out, br, ch) is a function that calls the encoder under test such that the waveform ‘in’ is encoded to the bitstream ‘out’ at the bitrate ‘br’ and using channelmode ‘ch’
– decoder_bandlimiting (in, out) is a function that calls the fixed-point encoder test tool such that it decodes the AAC part of the bitstream ‘in’ to the waveform ‘out’, while limiting the bandwidth to the AAC part and performing QMF-domain upsampling
– PEAQ_advanced (ref, deg) is a function that compares the reference waveform ‘ref’ with the degraded waveform ‘deg’ returning the ODG resulting from that comparison
The mean of the resulting vector odg_difference[cfg] shall not indicate an average performance of the encoder under test that is worse by more than 0.2. Moreover the worstcase difference shall not exceed 0.5. These criteria shall be fulfilled for all configurations.
6.2.2.2 SBR encoder conformance
Conformance for the SBR part of the encoder shall be measured for the various elements of the SBR bitstream independently. To eliminate the influence of potentially differing delays between the encoder under test and the reference encoder, the input waveforms shall be zero-padded at the beginning such that both encoders produce equivalent framing.
The fixed-point encoder test tool shall be used to first extract the various SBR parameters that are embedded in the SBR bitstream under test and subsequently compare them to the output of the reference encoder.
The following is an overview of how parameter extraction and parameter comparison work for the various modules. For further details, please consult the attached source code of the fixed-point encoder test tool:
– The envelope energies signaled by the encoder under test are combined to the equivalent envelope energies for one envelope. This is done after dequantization and M/S -> L/R unmapping. The comparison however takes place in the log-domain. For higher energies only low deviations are allowed, while lower energies are permitted to deviate more: Differences of up to 3dB are allowed for all energies, differences of up to 6dB are allowed for energies up to 42 dB below a full scale sine, differences of up to 9 dB are allowed for energies up to 60 dB below a full scale sine.
– The test of the transient detector is performed by comparing the bitstream element sbr_grid() from both encoders. The RMS of the difference between the transient position vector of the encoder under test and the reference encoder shall not be greater than 0.2.
– The test of the missing harmonics detector is carried out by comparing the bitstream elements sbr_sinusoidal_coding() from both encoders. The vector that is constructed by subtracting the missing harmonics vectors from both bitstreams and concatenating the result over all frames shall not have an RMS greater than 0.2.
– The test of the whitening level detector is performed by comparing the bitstream element sbr_invf() from both encoders. The vector that is constructed by subtracting the whitening level vectors of the encoder under test and the reference encoder shall not have an RMS greater than 0.2.
– The test for the noise floor estimator module is identical to the test for the envelope energy conformance test, except that the actual noise floor energies are compared instead of the envelope energies. The maximum permissible noise floor energy difference for a given deviation between reference encoder and encoder under test are as follows: Differences of up to 6 dB are allowed for all energies, differences of up to 9 dB are allowed for energies up to 42 dB below a full scale sine, differences of up to 12 dB are allowed for energies up to 60 dB below a full scale sine.
6.2.2.3 PS encoder conformance
All comparisons shall be limited to frames and bands in the bitstream that contain an energy greater than -87 dB below a full-scale sine. Prior to the actual conformance test, the stereo parameters from both encoders are mapped to the highest available stereo frequency resolution.
– the signal that is constructed from the differences between all IID values signaled by the encoder under test and the reference encoder shall have a maximum absolute difference of 2 and the RMS must not exceed 0.25
– the signal that is constructed from the differences between all ICC values signaled by the encoder under test and the reference encoder shall have a maximum absolute difference of 2 and the RMS must not exceed 0.25
6.2.3 Subjective criteria
Subjective tests shall cover the encoder configurations tested during the characterization phase. The requirement for passing the subjective tests is that the encoder under test does not perform worse in a statistically significant sense in any test case when compared to the fixed-point reference encoder.
Annex A:
Void
Table 1: Void
Annex B (normative):
Test vectors for fixed-point encoder conformance
Table 2: Testvectors specifically relevant for the objective testing of the AAC tool
|
# |
Test vector file name |
|
1 |
ct_castagnettes.wav |
|
2 |
ct_glockenspiel.wav |
|
3 |
ct_test_1_48.wav |
|
4 |
ct_test_2_48.wav |
|
5 |
ct_test_3_48.wav |
|
6 |
cymbal.wav |
|
7 |
gong.wav |
|
8 |
guitar_1.wav |
|
9 |
guitar_2.wav |
|
10 |
guitar_cymbals.wav |
|
11 |
handcuff.wav |
|
12 |
hihat.wav |
|
13 |
laugh.wav |
|
14 |
shaker.wav |
|
15 |
speech_female.wav |
|
16 |
speech_male_german.wav |
|
17 |
speech_male_portuguese.wav |
|
18 |
triangle.wav |
|
19 |
wooden_toys_1.wav |
|
20 |
wooden_toys_2.wav |
Table 3: Testvectors specifically relevant for the objective testing of the SBR tool
|
# |
Test vector file name |
SBR module |
|
21 |
noise_fade.wav |
Envelope |
|
12 |
hihat.wav |
Transients |
|
1 |
ct_castagnettes.wav |
Transients |
|
22 |
sines.wav |
Sines |
|
21 |
noise_fade.wav |
Noise |
|
23 |
invf.wav |
Whitening |
Table 4: Testvectors specifically relevant for the objective testing of the PS tool
|
# |
Test vector file name |
PS parameter |
|
3 |
ct_test_1_48.wav |
IID |
|
2 |
ct_glockenspiel.wav |
IID |
|
3 |
ct_test_1_48.wav |
ICC |
|
18 |
triangle.wav |
ICC |
Annex C (informative):
Change history
|
Change history |
|||||||
|
Date |
TSG SA# |
TSG Doc. |
CR |
Rev |
Subject/Comment |
Old |
New |
|
2005-06 |
28 |
SP-050242 |
Approved at TSG SA#28 |
2.0.0 |
6.0.0 |
||
|
2005-12 |
30 |
SP-050786 |
0001 |
2 |
Correction to Enhanced aacPlus Encoder conformance |
6.0.0 |
6.1.0 |
|
2006-03 |
31 |
SP-060013 |
0002 |
Correction to Enhanced aacPlus Decoder conformance: replacement of one test-vector |
6.1.0 |
6.2.0 |
|
|
2006-06 |
32 |
SP-060354 |
0004 |
1 |
Correction to Enhanced aacPlus Decoder conformance: clarification of conformance criteria |
6.2.0 |
6.3.0 |
|
2007-03 |
35 |
SP-070030 |
0005 |
1 |
Change of Enhanced aacPlus decoder conformance for the Parametric Stereo tool |
6.3.0 |
7.0.0 |
|
2008-12 |
42 |
Version for Release 8 |
7.0.0 |
8.0.0 |
|||
|
2009-12 |
46 |
Version for Release 9 |
8.0.0 |
9.0.0 |
|||
|
2011-03 |
51 |
Version for Release 10 |
9.0.0 |
10.0.0 |
|||
|
2012-09 |
57 |
Version for Release 11 |
10.0.0 |
11.0.0 |
|||
|
2014-09 |
65 |
Version for Release 12 |
11.0.0 |
12.0.0 |
|||
|
2015-12 |
70 |
Version for Release 13 |
12.0.0 |
13.0.0 |
|||
|
Change history |
|||||||
|
Date |
Meeting |
TDoc |
CR |
Rev |
Cat |
Subject/Comment |
New version |
|
2017-03 |
75 |
Version for Release 14 |
14.0.0 |
||||
|
2018-06 |
80 |
Version for Release 15 |
15.0.0 |
||||
|
2020-07 |
– |
– |
– |
– |
– |
Update to Rel-16 version (MCC) |
16.0.0 |
|
2022-04 |
– |
– |
– |
– |
– |
Update to Rel-17 version (MCC) |
17.0.0 |