9a Stereoscopic 3D video

26.2443GPP3GPP file format (3GP)Release 17Transparent end-to-end Packet-switched Streaming Service (PSS)TS

9a.1 General

Stereoscopic 3D video can be encapsulated and delivered in 3GP files (or 3GP segments in the case of DASH). Frame compatible H.264/AVC and temporally interleaved H.264/AVC use the AVC file format as specified in clause 5 of [20] where information about the stereo arrangement is carried in an SEI message "frame packing arrangement SEI". Multiview Video Coding (MVC) on the other hand uses the MVC file format as specified in clause 7 of [20] which specify separate signalling for MVC streams.

Storing frame compatible or temporally interleaved stereoscopic 3D video in a 3GP file as described above ensures that a UE can decode the bitstreams correctly (if it has the corresponding decoding capability), but it does not ensure that a UE renders the 3D video correctly. For instance, a UE that is not aware of the SEI message indicating that a bitstream represents frame compatible 3D or temporally interleaved 3D will simply render the video frames as consecutive 2D frames. The output will most likely look like garbage or with disturbing artefacts to the viewer.

The above problem is avoided by enforcing post-decoder requirements with the restricted video mechanism specified in the ISO base media file format [7]. The mechanism is similar to the content protection transformation where sample entries are hidden behind generic sample entries, ‘encv’, ‘enca’, etc., indicating encrypted or encapsulated media. The analogous mechanism for restricted video uses a transformation with the generic sample entry ‘resv’. The method should be applied when the content should only be decoded by clients that present it correctly. For the above cases with frame compatible and temporally interleaved 3D video, the scheme type for stereoscopic video ‘stvi’ [7] should be used.

In addition, UEs consuming content provided in the 3GP file format expect to identify the content based on the MIME Type of the 3GP file in order to accept or reject content. RFC6381 [34] provides an ability to signal profile and codec parameters and may be considered to be used in this context as well. For more details refer to clause 9a.5.

The following sub-clauses describe stereoscopic 3D file format signalling in more detail including the case of mixed services.

9a.2 Frame compatible H.264/AVC

Frame compatible H.264/AVC is stored in a 3GP file as defined for H.264/AVC in the AVC file format as specified in clause 5 of [20] where the AVC sample entry has been transformed according to the restricted video mechanism using the sample entry ‘resv’ and the stereo video scheme type ‘stvi’ [7]. The stereo scheme of the stereo video box is 1, i.e. the stereo indication type identifies the frame packing arrangement type by using the values defined by the frame packing arrangement SEI (Table D-8 of ISO/IEC 14496-10), for example 3 for Side by Side or 5 for temporal interleaved.

NOTE: It is recommended to use the "frame packing arrangement SEI" rather than the "stereo video SEI".

9a.3 Multiview Video Coding MVC

Multiview Video Coding MVC is stored and signalled in a 3GP file as defined for MVC in the MVC file format as specified in clause 7 of [20]. In order to ensure compatibility with H.264 (AVC) file readers, at least one track shall use the sample entry type ‘avc1’.

9a.4 Mixed 2D/3D video

Decoding and rendering requirements are signalled in the sample entry descriptions as detailed above. In fact, each video sample in a 3GP file is associated with a sample entry description, which in turn specifies if the video data is 2D H,264/AVC or any of the above types of 3D H.264/AVC. If a file contains both 2D and 3D video, separate sample entry descriptions are used where 2D parts of the file are associated with the 2D sample entry description and 3D parts of the file with the appropriate 3D sample entry description.

9a.5 MIME type signaling for 3D stereoscopic video files

UEs consuming content provided in the 3GP file format expect to identify the content based on the MIME Type of the 3GP file in order to accept or reject content. RFC6381 [34] provides an ability to signal profile and codec parameters and may be considered to may be used in this context as well.

To signal content provided in MVC, the codecs parameter as defined in RFC6381 [34] may be used. The details on how to signal MVC content are provided in RFC6381 [34], clause 3.3. The clause addresses also the use case when MVC content is coded in an H.264/AVC-compatible fashion.

In case of mixed content, all required capabilities may be signalled in the MIME type parameters.