6 AI/ML management use cases and requirements

28.1053GPPArtificial Intelligence/ Machine Learning (AI/ML) managementManagement and orchestrationRelease 17TS

6.1 General

The use cases and requirements for AI/ML management are specified in the following clauses.

6.2 ML training

6.2.1 Description

In operational environment before the ML entity is deployed to conduct inference, the ML model associated with the ML entity needs to be trained (e.g. by ML training function which may be a separate or an external entity to the AI/ML inference function).

NOTE: In th present document, ML entity training refers to ML model training associated with an ML entity.

The AI/ML Entity is trained by the ML training (MLT) MnS producer, and the training can be triggered by request(s) from one or more MLT MnS consumer(s), or initiated by the MLT MnS producer (e.g. as result of model evaluation).

6.2.2 Use cases

6.2.2.1 ML training requested by consumer

The ML training capabilities are provided by an AIMLT MnS producer to one or more consumer(s).

Figure 6.2.2.1-1: ML training requested by MLT MnS consumer

The ML training may be triggered by the request(s) from one or more MLT MnS consumer(s). The consumer may be for example a network function, a management function, an operator, or another functional differentiation To trigger an ML training, the AIMLT MnS consumer requests the MLT MnS producer to train the ML model associated with an ML enabled function. In the ML training request, the consumer should specify the inference type which indicates the function or purpose of the ML entity, e.g. CoverageProblemAnalysis. The MLT MnS producer can perform the training according to the designated inference type. The consumer may provide the data source(s) that contain(s) the training data which are considered as inputs candidates for training. To obtain the valid training outcomes, consumers may also designate their requirements for model performance (e.g. accuracy, etc) in the training request.

The MLT MnS producer provides a response to the consumer indicating whether the request was accepted.

If the request is accepted, the MLT MnS producer decides when to start the ML training with consideration of the request(s) from the consumer(s). Once the training is decided, the producer performs the followings:

– selects the training data, with consideration of the consumer provided candidate training data. Since the training data directly influences the algorithm and performance of the trained ML Entity, the MLT MnS producer may examine the consumer’s provided training data and decide to select none, some or all of them. In addition, the MLT MnS producer may select some other training data that are available;

– trains the ML entity using the selected training data; and

– provides the training results (including the location of the trained ML model or entity, etc.) to the MLT MnS consumer(s).

6.2.2.2 ML training initiated by producer

The ML training may be initiated by the MLT MnS producer, for instance as a result of performance evaluation of the AI/ML model, based on feedback or new training data received from the consumer, or when new training data which are not from the consumer describing the new network status/events become available.

When the MLT MnS producer decides to start the ML training, the producer performs the followings:

– selects the training data;

– trains the ML entity using the selected training data; and

– provides the training results (including the location of the trained ML entity, etc.) to the MLT MnS consumer(s) who have subscribed to receive the ML training results.

6.2.2.3 ML model and and ML entity selection

For a given machine learning-based use case, different entities that apply the respective ML model or AI/ML inference function may have different inference requirements and capabilities. For example, one consumer with specific responsibility and wish to have an AI/ML inference function supported by an ML model or entity trained for city central business district where mobile users move at speeds not exceeding 30 km/hr. On the other hand, another consumer, for the same use case may support a rural environment and as such wishes to have an ML model and AI/ML inference function fitting that type of environment. The different consumers need to know the available versions of ML entities, with the variants of trained ML models or entities and to select the appropriate one for their respective conditions.

Besides, there is no guarantee that the available ML models/entities have been trained according to the characteristics that the consumers expect. As such the consumers need to know the conditions for which the ML models or ML entities have been trained to then enable them to select the models that are best fit to their conditions and needs.

The models that have been trained may differ in terms of complexity and performance. For example, a generic comprehensive and complex model may have been trained in a cloud-like environment but when such a model cannot be used in the gNB and instead, a less complex model, trained as a derivative of this generic model, could be a better candidate. Moreover, multiple less complex models could be trained with different level of complexity and performance which would then allow different relevant models to be delivered to different network functions depending on operating conditions and performance requirements. The network functions need to know the alternative models available and interactively request and replace them when needed and depending on the observed inference‑related constraints and performance requirements.

6.2.2.4 Managing ML training processes

This machine learning capability relates to means for managing and controlling ML model/entity training processes.

To achieve the desired outcomes of any machine learning relevant use-case, the ML model applied for such analytics and decision making, needs to be trained with the appropriate data. The training may be undertaken in managed function or in a management function.

In either case, the network (or the OAM system thereof) not only needs to have the required training capabilities but needs to also have the means to manage the training of the ML models/entities. The consumers need to be able to interact with the training process, e.g. to suspend or restart the process; and also need to manage and control the requests related to any such training process.

6.2.2.5 Handling errors in data and ML decisions

Traditionally, the ML models/entities (e.g. ML entity and ML entity) are trained on good quality data, i.e. data that were collected correctly and reflected the real network status to represent the expected context in which the ML entity is meant to operate. Good quality data is void of errors, such as:

– Imprecise measurements, with added noise (such as RSRP, SINR, or QoE estimations).

– Missing values or entire records, e.g. because of communication link failures.

– Records which are communicated with a significant delay (in case of online measurements).

Without errors, an ML entity can depend on a few precise inputs, and don’t need to exploit the redundancy present in the training data. However, during inference, the ML entity is very likely to come across these inconsistencies. When this happens, the ML entity shows high error in the inference outputs, even if redundant and uncorrupted data are available from other sources.

Figure 6.2.2.5-1: The propagation of erroneous information

As such the system needs to account for errors and inconsistencies in the input data and the consumers should deal with decisions that are made based on such erroneous and inconsistent data. The system should:

1) enable functions to undertake the training in a way that prepares the ML entity to deal with the errors in the training data, i.e. to identify the errors in the data during training; and

2) enable the MLT MnS consumers to be aware of the possibility of erroneous input data that are used by the ML entity.

6.2.3 Requirements for ML training

Table 6.2.3-1

Requirement label	Description	Related use case(s)
REQ-ML_TRAIN-FUN-01	The MLT MnS producer shall have a capability allowing the consumer to request ML training.	ML training requested by consumer (clause 6.2.2.1)
REQ- ML_TRAIN-FUN-02	The MLT MnS producer shall have a capability allowing the consumer to specify the data sources containing the candidate training data for ML training.	ML training requested by consumer (clause 6.2.2.1)
REQ- ML_TRAIN-FUN-03	The MLT MnS producer shall have a capability allowing the consumer to specify the inference type of the ML model entity to be trained.	ML training requested by consumer (clause 6.2.2.1)
REQ- ML_TRAIN-FUN-04	The MLT MnS producer shall have a capability to provide the training result (including the location of the trained ML model entity) to the consumer.	ML training requested by consumer (clause 6.2.2.1), /ML training initiated by producer (clause 6.2.2.2)
REQ-ML_SELECT-01	3GPP management system shall have the capability to enable an authorized consumer to discover the characteristics of available models including the contexts under which each of the models was trained.	ML model and ML entity selection (clause 6.2.2.3)
REQ-ML_SELECT-02	3GPP management system shall have the capability to enable an authorized consumer to select an ML model.	ML models and ML entity selection (clause 6.2.2.3)
REQ-ML_SELECT-03	The MLT MnS producer shall have the capability to enable an authorized consumer to request for a model to be trained to satisfy the consumer’s expectations.	ML training requested by consumer (clause 6.2.2.1), ML model and ML entity selection (clause 6.2.2.3)
REQ-ML_SELECT-04	3GPP management system shall have the capability to enable an authorized consumer to request for information and be informed about the available alternative models of differing complexity and performance.	ML model and ML entity selection (clause 6.2.2.3)
REQ-ML_SELECT-05	3GPP management system shall have the capability to enable an authorized consumer to request one of the known or available alternative models of differing complexity and performance to be used for inference.	ML model and ML entity selection (clause 6.2.2.3)
REQ-ML_SELECT-06	The 3GPP management system shall have a capability to provide a selected ML model/entity to the consumer.	ML model and ML entity selection (clause 6.2.2.3)
REQ-ML_TRAIN- MGT-01	The MLT MnS producer shall have a capability allowing an authorized consumer to manage and configure one or more requests for the training of specific ML models or ML entities, e.g. to modify the characteristics of the request or to delete a request.	ML training requested by consumer (clause 6.2.2.1),Managing ML Training Processes (clause 6.2.2.4)
REQ-AIML_TRAIN- MGT-02	The MLT MnS producer shall have a capability allowing an authorized consumer to manage and configure one or more training processes, e.g. to start, suspend or restart the training; or to adjust the training conditions and/or characteristics.	ML training requested by consumer (clause 6.2.2.1), Managing ML training processes (clause 6.2.2.4)
REQ-ML_TRAIN- MGT-03	3GPP management system shall have the capability to enable an authorized consumer (e.g. the function/entity different from the function that generated a request for ML model/entity training) to request for a report on the outcomes of a specific training instance.	Managing ML training processes (clause 6.2.2.4)
REQ-ML_TRAIN- MGT-04	3GPP management system shall have the capability to enable an authorized consumer to define the reporting characteristics related to a specific training request or training instance.	Managing ML training processes (clause 6.2.2.4)
REQ-ML_TRAIN- MGT-05	3GPP management system shall have the capability to enable the MLT function to report to any authorized consumer about specific ML Training process and/or report about the outcomes of any such ML training process.	Managing ML training processes (clause 6.2.2.4)
REQ-ML_ERROR-01	The 3GPP management system shall enable an authorized consumer of data services (e.g. an MLT function) to request from a producer of data services a Value Quality Score of the data, which is the numerical value that represents the dependability/quality of a given observation and measurement type.	Handling errors in data and ML decisions (clause 6.2.2.5)
REQ-ML_ERROR-02	The 3GPP management system shall enable an authorized consumer of AI/ML decisions (e.g. a controller) to request ML decision confidence score which is the numerical value that represents the dependability/quality of a given decision generated by an AI/ML-inference function.	Handling errors in data and ML decisions (clause 6.2.2.5)
REQ-ML_ERROR-03	The 3GPP management system shall enable a producer of data services (e.g. a gNB) to provide to an authorized consumer (e.g. an MLT function) a Value Quality Score of the data, which is the numerical value that represents the dependability/quality of a given observation and measurement type.	Handling errors in data and ML decisions (clause 6.2.2.5)
REQ-ML_ERROR-04	The 3GPP management system shall enable a producer of ML decisions (e.g. an AI/ML inference function) to provide to an authorized consumer of AI/ML decisions (e.g. a controller) an AI/ML decision confidence score which is the numerical value that represents the dependability/quality of a given decision generated by the inference function.	Handling errors in data and ML decisions (clause 6.2.2.5)