7 Test Methodologies for Immersive Audio Systems of TS 26.118 (Codec Quality Characterization Test with Binaural Rendering)

26.2593GPPRelease 17Subjective test methodologies for the evaluation of immersive audio systemsTS

7.1 Introduction

This clause specifies the optional, but strongly recommended, codec quality characterization test for the audio profiles in TS 26.118 with binaural rendering over headphones. The Codec Quality Characterization test with Binaural Rendering is based on the test method defined in [2].

7.2 Experimental Design

The experimental design of the Codec Quality Characterization Test with Binaural Rendering is such that all assessors rate all test Conditions. To control for possible presentation order biases, the presentation order of the test materials is fully randomized during the experiment (double-blind test). To minimize listener fatigue, the following constraints on the experimental design are defined:

– Each Test Material shall be no longer than 12 s in duration.

– No more than four Codec Operating Points shall be tested for each Test Material.

– Each experiment shall contain no more than 10 Test Materials.

7.3 Selection of Assessors

The selection of assessors shall follow the guidelines in [2] clause 4.1. Only experienced assessors shall participate in the experiment and the test administrator shall employ pre- and post-screening according to [2] clause 4.1. The final test results shall include assessments from at least 10 experienced assessors that have passed both pre- and post-screening.

7.4 Test Materials

Critical audio materials representing typical virtual reality content shall be used for this test. Each test should include at least 3 channel-based, 3 object-based and 3 scene-based Test Materials and no more than 10 Test Materials in total.

All Test Materials shall be provided as either 24-bit integer or 32-bit PCM float signals with a sampling rate of 48 kHz.

7.5 Content Presentation

The content presentation and grading process are according to [2] clauses 5.3 and 5.4.

7.6 Listening Environment

For each octave-band, the maximum sound pressure level of the listening environment shall not exceed the levels in Table 2 (corresponding to an NR20 noise rating curve):

Table 2: Maximum Sound Pressure Level for Listening Environment

Octave Band centre frequency	31.5 Hz	62.5 Hz	125 Hz	250 Hz	500 Hz	1 kHz	2 kHz	4 kHz	8 kHz
Maximum Sound Pressure Level (dBSPL)	69	51	39	31	24	20	17	14	13

7.7 Listening System

The listening system shall be headphone-based using the Common Informative Binaural Renderer (CIBR) for both the Reference and Degraded conditions. The CIBR is described in [5].

The binauralization shall use either individualized HRTFs or HRTFs based on a head and torso simulator (HATS). The choice of HRTF set shall be indicated in the test report. The headphones shall be equalized. If individualized HRTFs are used, the headphones shall have individualized equalization. If HATS HRTFs are used, the headphones shall be equalized for the same make/model of HATS.

7.8 Listening Level

The listening level is according to [2] clause 8. The listening level is adjusted with channel-based content.

7.9 Anchor/Reference Conditions

All Codec Quality Characterization Tests shall include one Hidden Reference and two Anchors. The two Anchors are 3.5kHz and 7kHz low-pass filtered versions of the Reference condition, as described in [2] clause 5.1.

The Reference and Hidden Reference conditions are the source test Materials binaurally rendered to headphones through the Common Informative Binaural Renderer (CIBR) described in [5].

7.10 Test Conditions

The Test Conditions are generated by encoding, decoding and rendering the test Materials with the target operating points of:

– 128 kbps (for First Order Ambisonics contents only)

– 256 kbps

– 384 kbps

– 512 kbps

A +/- 10 % variation from the target operating points is acceptable. The actual bit-rate for each Test Condition shall be reported with an accompanying justification for the target operating point deviation. The renderer used for the Test Conditions shall be the same renderer used for the Anchor and Reference Conditions.

7.11 Attributes

The Codec Quality Characterization Test with Binaural Rendering shall assess the Basic Audio Quality attribute described in [2] clause 6.4.

7.12 Test Report and Presentation of Results

The test report shall provide the Mean and 95% Confidence Intervals (t-distribution) for each test Condition, Hidden Reference and Anchors. All results provided shall be post-screened results (see clause 7.3).

Annex A (informative):
Change history

Change history
Date	Meeting	TDoc	CR	Rev	Cat	Subject/Comment	New version
09-2018	SA#81	SP-180643				Presented to TSG SA#81 for approval	1.0.0
09-2018	SA#81					Approved at TSG SA#81	15.0.0
2020-07	–	–	–	–	–	Update to Rel-16 version (MCC)	16.0.0
2022-04	–	–	–	–	–	Update to Rel-17 version (MCC)	17.0.0