Allan HENRY

Steering a drone by voice requires understanding what a user is saying. Existing voice-control systems rely on predefined lexicons, and no resources document unconstrained speaker speech. This thesis starts from this observation and addresses it in three steps: characterizing spontaneous vocal guidance, producing a corpus aligned with flight commands, and designing an architecture capable of interpreting speech in real time to generate drone movements.

Two requirements structure the problem. Making piloting accessible to untrained users implies accepting unconstrained speech. Maintaining control stability imposes a fast response: beyond a 200 ms delay between intention and reaction, the operator is correcting a situation that has already passed, leading to instability. After deducting fixed acquisition and transmission delays, the speech understanding budget is approximately 100 ms. Literature addresses one or the other of these requirements, but never both.

Three experiments were conducted using a shared dyadic protocol, where a guide vocally directs a pilot toward targets the pilot cannot see. In the first, the pilot is a human moving blindfolded, in the second, they control a drone via a joystick, in the third, an automated system replaces the pilot without the guide's knowledge. Corpus analysis reveals four properties of spontaneous guidance: less than 2% of utterances combine multiple directions (guides break navigation down into elementary instructions), just over half carry an explicit directional marker (immediate context brings this to 61%, with the remainder covering flow control), movement amplitude, rarely quantified, is modulated by mitigators (e.g., "a little") correlated with distance to target rather than movement duration, finally, guidance unfolds in three phases: Orientation, Approach, and Adjustment. It remains identical across both human configurations despite very different motor interfaces.

This resulted in the VoiceStick corpus, the first French spontaneous speech resource aligned with flight control commands: 4,219 utterances produced by 29 dyads, each paired with two labels (the guide's intention and the executed action). This duality allows measuring the gap between what is asked and what is performed.

Architecturally, a cascaded approach is ruled out: 202 ms inference with 81% of errors attributable to transcription. The proposed end-to-end architecture, based on a French self-supervised encoder with weighted layers and attention aggregation, achieves 93% intention accuracy in 7 ms. A duration regression model based solely on temporal metadata leverages a novel finding: silence between utterances predicts movement amount better than any prosodic cue. Adapted for continuous streaming, the system processes 300 ms frames in 60 ms and reaches 70% accuracy in deployment, with degradation affecting direction rather than movement.

Flight validation, where ten participants alternately direct a human pilot or the system without knowing which, reveals no difference across nine measured user experience dimensions, with half of the evaluations assigning identical scores to both pilots. This work demonstrates that a system trained on unconstrained speech can operate in real time, and that critical design constraints are not derived from intuition: they can only be observed in data that had never been collected before.