Conversational agents in intelligent environments by using the Home Assistant platform
Author: Aleks Brenčič
Mentor: izr. prof. dr. Matej Rojc univ. dipl. inž. el.
Degree: 1.
Date: september, 2020
DKUM: ALEKS BRENČIČ
The aim of the project is to develop a system for IoT environments that would include a conversational interface for advanced interaction between the user and the smart home control system.
The system includes a communication node that connects devices over a Wi-Fi network. Communication with the node takes place via the MQTT protocol. The central control system is the Home Assistant platform, which runs on a virtual machine running Linux. To communicate between the Home Assistant platform and the user, we have developed a multimedia interface (graphic elements and voice commands), through which the user can observe and voice control devices using the Almond dialog system.
Virtual assistant is a conversational interface, which can be used for advanced interaction solutions between IoT devices and users, and thus simplify and optimize increasingly complex services to users.
Assistants are used in a variety of ways today: from mobile apps, websites, all the way to smart speakers. The diverse ability to communicate with devices depends on the complexity of the conversational interface, where the simplest conversational interfaces support the use of commands or phrases, with or without a response from the device. Recently, task-oriented dialogue systems have also become more interesting, including smart speakers such as Amazon Alexa, Apple Siri, Cortana or Google home, all of which are designed to perform certain tasks or have short conversations, to obtain information from the user, and to assist in the performance of certain tasks. These include managing smart devices in the home, weather inquiries, playing music, sending messages, etc. The core of such systems uses AI solutions (artificial intelligence), i.e. algorithms and machine-learned models that mimic human cognitive behavior.
The Home Assistant (HA) platform is a framework used to manage and control devices in IoT environments.
It is an open source system, which can basically be a local stand-alone system, but it also has some additional functionalities available in cloud deployments. The framework can be run on an embedded system (e.g. Rasberry Pi), or also on virtual systems (e.g. VMware), which can then be accessed and configured via the Internet or the console. The programming language with which we configure it is Python. There is also a mobile version by which we can also monitor and manage all devices connected to the HA system outside the home. The Home assistant platform includes a wide range of functionalities.
IoT system development with Almond conversational assistant.
Almond is an open source virtual assistant used for online services and IoT systems. Nowadays, it is already very widespread because it has a wide range of functionalities and is user-friendly. It consists of three main components: Thingpedia, ThinkTalk and ThingSystem. Thingpedia is used for applications for IoT environments. Thus, it is used for devices such as light bulbs, or thermostats, etc. Each device has an entry that feeds the natural language interface, and that represents how people refer, participate, and influence devices. Thingtalk is a high-level programming language developed to connect to the Internet of Things. It connects the API (application programming interface) from devices together, while hiding details from device configuration and communication. ThingSystem is a component that includes all the information each user keeps about their devices and commands. It is portable, which means users can use it on their mobile platforms or on the cloud. ThingSystem has two main tasks: to assist the user in configuring and managing his devices, and to execute program code.
Diskussion and results
