Yongxin Zhou

Abstract:

This thesis explores Natural Language Generation (NLG) in digital therapy, focusing on the generation of reports that summarize cognitive training sessions conducted by patients at home with a virtual assistant. The aim is to provide therapists with valuable insights for patient follow-up. We addressed two main objectives: identifying relevant medical information to report and extracting key points from patient-virtual assistant dialogues.

Conducted in low-resource settings, we faced challenges such as expert unavailability, lack of example texts and the handling of affective information. To address these challenges, we adopted two strategies: Data-to-Text (D2T) and Text-to-Text (T2T) generation, with a focus on affect-aware NLG.

Regarding T2T generation, we evaluated the ability of various pre-trained language models to summarize dialogues. We evaluated their performance using automatic measures and human evaluation. In addition, we introduced a new metric, PSentScore, to measure the preservation of affective content in dialogue summaries. Our results showed that existing summarization models often neglect affective content, but careful training target selection can reduce this mismatch.

Regarding D2T generation, we presented a real-life application of NLG systems: the generation of reports summarizing remote remediation sessions. We developed a template-based system and conducted experiments with GPT-4. We carried out a human evaluation which showed the effectiveness of the proposed systems and shed light on ways to further improve the generation of these reports.

This thesis contributes to the field of NLG evaluation and application, taking the first steps towards affect-aware NLG, crucial for domains like healthcare and customer service. Our research highlights the importance of incorporating emotions and sentiments into NLG systems to improve their effectiveness in real-world applications, such as aiding speech therapists with automated tools to generate reports and summarize patient progress.