Use the right AI for the right need at the right time

• AI
• CSR
• Education
• Prospective
• Technology

Share on Twitter : Use the right AI for the right need at the right time (New window) Share on Linkedin : Use the right AI for the right need at the right time (New window)

Life cycle of an AI system

Lifecycle Assessment (LCA) [7] is a systematic approach to evaluate the environmental impacts of a product or system throughout its entire life cycle [1]. The life cycle of an AI system is similar to the previous “life cycle of data mining project” [8]. Indeed, as for data mining, the AI lifecycle encompasses the complete process of developing and deploying artificial intelligence systems. It starts with data collection and moves through stages such as data preprocessing, model training, evaluation, deployment, and ongoing monitoring and maintenance.

There is no universal method for finding the right trade-off, however one way is to use benchmarking, which plays a crucial role in the development of frugal AI by improving efficiency and adaptability.

Due to the life cycle of an AI system, many costs prevent AI from being frugal.

A non-exhaustive list can be found in the table below:

These costs can accumulate and impact the overall frugality of an AI system, as detailed in recent publications e.g. [11]. The cost to pay is the addition of these costs, and some of them are recurrent costs that must be paid at every use of a given model as for example the inference cost. The cost to pay is not, as assumed in some publications, only the costs induced in the three steps: training, deployment, and production. We encourage considering the sum of all the costs of the life cycle of an AI system and not only part of them.  For instance, the fine-tuning of an existing model only reduces one of the costs (the training cost). Even when the model must be only updated and not replaced by a new one, frugality should be considered and updating the model is an investment decision which, as in the financial markets, should be taken only if a certain return on investment is expected [12].

In summary, given a task to be solved by AI, the total costs should be minimized and the ROI should be considered.

Large Models are not always the best option for a given task

The recent trend in AI is the use of large models (generative AI, large deep neural networks, etc.). Many of the tasks that could be performed with AI ( classification , regression , etc.) are currently not frugally solved by large models.

Therefore, one should keep in mind that “old models/no large models”  remains quite interesting in terms of performances particularly on tabular data or time series, as illustrated below on sentiment analysis. Examples of ‘no large models’ are Linear Regression, K-nearest neighbors, Random Forest [4], Catboost [10], Khiops [2], etc., or even signal processing for time series (as, for example, exponential smoothing, Arima, etc. [3]).

Finding the right inflection point between performance and frugality

Finding the right inflection point between performance and frugality indicators in AI models is critical to maximizing efficiency, accessibility, and ethical considerations, while still achieving satisfactory levels of performance. Balancing these factors can lead to more sustainable and impactful AI solutions. Besides simplification gains, there are many arguments in favour of finding the right tipping point [3], the more obvious ones being the improvement of:

Resource efficiency:

• Cost reduction: Energy-efficient models require less computing power and memory, resulting in lower operating costs.
• Environmental impact: Reducing resource consumption can reduce the carbon footprint associated with training and deploying AI models.

Scalability:

• Broader accessibility: More efficient models can be deployed in resource-constrained environments, making AI accessible to a wider audience.
• Faster deployment: More efficient models can be trained and deployed faster, allowing rapid iteration and adaptation.

Optimized Performance:

• Diminishing returns: At a certain point, increasing model complexity yields minimal performance gains. Identifying the tipping point helps avoid unnecessary complexity.
• Robustness: Simpler models can sometimes generalize better to unseen (test or deployment) data, reducing the risk of overfitting (learning particularities of the training data that will be not present when using later the model on test or deployment data)

User Experience:

• Latency reduction: Frugal models often result in faster inference times, improving the user experience in real-time applications.
• Ease of integration: Less complex models can be more easily integrated into existing systems and workflows.

Ethical aspects:

• Fairness and transparency: Simpler models can be more interpretable, making them easier to understand the decisions made by AI systems and promoting fairness.
• Bias mitigation: Frugal models can reduce the risk of embedding biases that can result from overly complex architectures.

Creativity issued from frugal innovation and experimentation:

• A focus on frugality can inspire innovative approaches to problem solving, leading to novel solutions that may not rely on heavy computational resources.

This list is not exhaustive, of course, and we can add costs that are sometimes ‘hidden’, such as increasing the skills of teams, integrating an additional data scientist into the project team, etc.).

One way to find this trade-off is to use benchmarking [6], which plays a crucial role in the development of frugal AI by improving efficiency and adaptability.
The results of benchmarking AI methods help to develop more frugal AI in several ways. Firstly, it is possible to identify efficient methods, since benchmarks enable comparing the performance of different AI methods, highlighting those that offer the best value for money in terms of the resources used. Secondly, it is possible to optimize resources: through analysis of the results, researchers (i.e. users) can identify algorithms that require less data or computing power, thus favoring lighter solutions. They also provide a consistent framework to evaluate AI models, ensuring comparability across different approaches (standardization). They help identify the most efficient algorithms for specific tasks, guiding resource allocation (performance metrics). They encourage sharing of best practices and datasets, fostering innovation in frugal AI solutions (community Collaboration).

The aim of benchmark results is not to systematically compare solutions (by repeating a lot of experiments), but to build up a set of skills that will enable an appropriate selection to be made. The question is therefore “how can companies that do not have data scientists build up this knowledge” (or companies that have qualified data scientists but who are overloaded with work and therefore cannot respond to all requests, etc.).

Illustration of the different possible tradeoff

As far as we know there is no universal method for finding the right tipping point. Modestly, however, we can mention one that makes sense at the start of a data science project:
(i) define the performance criterion for the project;
(ii) define the value of this criterion (perhaps in the form of a return on investment (ROI));
(iii) use a rule, an AI, etc., that is simple at the start and then, if the value of the criterion is not reached, make the AI more complex;
(iv) stop as soon as the value of the criterion is reached or when the sum of the costs becomes too important (or if the return on investment cannot be achieved or if the cost of achieving it will be too high).

Figure 1: Illustration of different tradeoffs between performances and costs

This is illustrated in Figure 1 above: In the purple case, if the return on investment in terms of performance is achieved with P1, there is no reason to make the AI model more complex and pay additional costs. In the green case, the same performance can be achieved for two different costs. It is therefore very interesting to start by using an AI model producing cost C1 and then stop. The worst case is where using an AI model produces a higher overall cost with poorer performance (not illustrated in the figure).
This last scenario is well presented in [9]. In this report a classification task is designed on text (sentiment analysis) using a Support Vector Machine (SVM) [5] or three Large Language Models (LLM) (BERT fine-tuned on the problem to solve, Llamma and BERT prompted to solve the problem). For this given classification task, we may observe that the biggest LLM energy consumptions for inference are several orders of magnitude higher than a standard SVM for a comparable (or lower) accuracy.