Duration: 09/09/2019 – 08/09/2023
Type of project: COST project
Chair: Isabelle Augenstein, Denmark
MC on UM FERI: assoc. prof. dr. Matej Rojc
Link: COST CA18231
Language generation (LG) is a crucial technology if machines are to communicate with humans seamlessly using human natural language. A great number of different tasks within Natural Language Processing (NLP) are language generation tasks, and being able to effectively perform these tasks implies (1) that machines are equipped with world knowledge that can require multi-modal processing and reasoning (e.g. textual, visual and auditory inputs, or sensory data streams), and (2) the study of strong, novel Machine Learning (ML) methods (e.g. structured prediction, generative models), since virtually all state-of-the-art NLP models are learned from data. Moreover, human languages can differ wildly in their surface realisation (i.e. scripts) as well as their internal structure (i.e. grammar), which suggests that multilinguality is a central goal if machines are to perform seamless language generation. Language generation technologies would greatly benefit both public and private services offered to EU citizens in a multilingual Europe, and have strong economic and societal impacts.Natural Language Generation (NLG) encompasses all Natural Language Processing (NLP) tasks that deal with automatically (or semi-automatically) generating readable text. Texts generated by NLG models usually have a large audience and can be quite application-specific. Those applications can be categorised in two broad areas: text-to-text and data-to-text generation. The former takes existing texts as input and produces a coherent text as output. Example applications include e.g. question answering (QA), converting nonlinguistic data to text, for instance, football commentary, weather and financial report generation, patient information summarization, and image/video caption generation.
https://dsplab.feri.um.si/wp-content/uploads/research-activities-2019-avatars-Medium.jpg
768
815
Matej
https://dsplab.feri.um.si/wp-content/uploads/dsplab_color.png
Matej2021-09-14 18:59:482021-09-15 15:52:11Gesture generation for 3D virtual agents and humanoid robotsDialogue, interaction and conversational language generation applications
HCI through natural language conversation is currently one of the most active research areas, where new commercial applications are emerging virtually every day. The so-called artificial intelligence paradigm (in most cases deep learning) is often used in the dialogue-based problem solving. Although, at present, the achievements are promising and many global companies and universities have focused their HCI efforts, the challenges of truly intelligent natural language understanding (NLU), and appropriate NLG regarding response appropriateness and natural outputs have not yet been addressed. The CA18231 will go beyond state-of-the-art technology in this field, using LG models in HCI task interaction for many interesting and challenging real-world use cases, such as conversational search interfaces; grounded dialogue models; real-time dialogue models; and conversational robots. Due to the complexity of NLU and NLG, much research focuses on the development of end-to-end systems. However, these are highly dependent on the availability of relevant data for learning. The lack of the required dataset (especially multimodal ones) limits the deep learning research possibilities for HCI. CA18231 project will solve this problem by construction of multilingual multimodal datasets to evaluate multimodal conversational interfaces. The current methods have not yet solved the problem of having a truly natural conversation. One main issue here is the difficulty of evaluating NLG systems since intrinsic evaluation metrics (e.g. BLEU, ROUGE) are often a poor proxy for model performance (Belz, 2009). This CA18231 will address this issue first by stressing the need for extrinsic evaluation. Concretely, dialogue agents will be tested by industry members in controlled environments.
ARRS J2-1737 HUMANIPA
Project group members
Periodic meeting for COST action CA18231
1st MC MEETING – COST KICK OFF MEETING