Alexandre AUDIBERT

Deep learning applications are rapidly expanding and show no signs of slowing down; they are even being used in more challenging domains such as medicine or law. The most critical com- ponent of a deep learning approach is pre-training a high-capacity model on a vast amount of data. This general pre-training approach is followed by a fine-tuning step, where the model is fine-tuned for a specific task. However, training these new types of models is very time-consuming, espe- cially when multiple tasks need to be solved, as this may require learning several separate models. Multi-task learning is a deep learning paradigm that seeks to learn multiple tasks simultaneously using only one model. This thesis focuses on studying task interactions and developing new al- gorithms for multi-task learning, particularly in the context of predictive justice. In our work we combined multi-task learning with low-rank tensor representation to improve parameter efficiency and to study task interactions. We have studied the multi-label text classification task, which is the most common multi-task scenario where the objective is to predict all labels associated with a given text. In our work, we propose to explore the interaction between labels through an advanced study of contrastive learning. Finally, we will address future work by studying the compatibility of the mixture of experts approach with multi-label tasks.

Our approaches are currently being tested on multiple datasets and show promising results for future research.