Ali Aghababaei Harandi

 

This thesis addresses a central challenge in modern deep learning: as models grow more capable, their dependence on massive computational resources and large annotated datasets makes them expensive to train and difficult to deploy in practice. This creates two distinct but related bottlenecks. On the one hand, the size of over-parameterized models leads to high memory usage and latency during inference. On the other hand, the constant need for human-labeled data remains a major limitation for building scalable systems. Rather than addressing hardware constraints and data limitations as separate problems, this work develops a unified approach to resource-efficient deep learning, arguing that practical artificial intelligence requires joint progress in both model compression and data-efficient learning.

 

To tackle the hardware bottleneck, we introduce RENE, a rank-adaptive tensor decomposition framework designed to compress pre-trained neural networks. RENE overcomes the challenge of manual tensor rank selection by treating it as a continuous, task-aware optimization process directly within the network's computational graph. Instead of relying on manual tuning or fixed constraints, the algorithm dynamically allocates its rank budget, preserving critical information in sensitive layers while compressing less critical components. Evaluated across both convolutional and transformer architectures, our model achieves a strong trade-off between compression rate and accuracy, demonstrating that theoretical reductions in parameters and FLOPs can successfully yield real inference speedups on edge devices.

 

To overcome the data supervision bottleneck, we present JEPAMatch, a semi-supervised learning framework that merges adaptive pseudo-labeling with geometric representation learning. To mitigate confirmation bias in standard self-training, JEPAMatch decouples discrete classification from the continuous geometric structure of the latent space. By combining Joint-Embedding Predictive Architectures (JEPA) with adaptive curriculum thresholding, the framework improves class balance and accelerates convergence, especially when labeled data is scarce. Beyond empirical results, this work explores the theoretical dynamics of self-training, suggesting that bounding the mutual information between consecutive training steps can promote convergence and act as a stabilizing mechanism.

 

By bridging formal theoretical analysis with practical algorithms, we show that adaptive, compact, and data-efficient strategies can preserve high performance while significantly accelerating convergence and inference speed. Ultimately, this work demonstrates that reducing computational costs and the dependence on human annotation is essential for making the next generation of deep learning algorithms more affordable and widely accessible.