Adrien Pupier

In recent years, speech processing and natural language processing methods have increasingly converged through the use of pre-trained models, driven in large part by advances in deep learning. This thesis falls within this context and explores a classical task in natural language processing — dependency parsing — applied to spontaneous speech, by working directly from the raw audio signal, without any intermediate transcription.

The objective is to show that acoustic cues present in speech can help better capture the specificities of spoken language, which are often poorly handled by traditional text-based models. Unlike pipeline approaches (transcription followed by parsing), our direct approach leverages these cues to improve the robustness of syntactic analysis.

Experiments conducted on spoken French demonstrate the relevance of this approach, as well as the crucial role played by the quality of pre-trained audio representations, particularly their exposure to spontaneous speech. This work also highlights specific challenges related to syntactic encoding in audio and to speech segmentation, which emerge as major bottlenecks for performing linguistic tasks directly from the audio signal.