Smart speakers in IoT environments

Smart speakers provide many useful features, such as capacitive keys for voice control, Bluetooth or Wi-Fi wireless connectivity, light response or ambient lighting, conversational agent, control over home automation of all devices, etc. With all the functionality, we can significantly increase the users’ quality of life in real environments. The system of using smart speakers in multi-room buildings is presented. In the system we have provided the playback and capturing of sound in several rooms. We also discussed, where processing is performed and what algorithms these speakers use. Among these, we were particularly interested in the speech recognition algorithm. In the system we envisioned connecting smart speakers to the Home Assistant framework.  

 

A smart speaker is an electronic device that can connect to conversational agents, which are usually cloud-based. The smart speaker captures the user’s speech, processes it in real time and also returns a response in the form of speech. There are several smart speakers from different manufacturers on the market today, and each has its advantages and disadvantages. Among the most popular are: Amazon Echo, Google Home, Apple HomePod, Sonos, etc. The greatest added value of smart speakers is the performance of the associated conversational assistants. The smart speaker is part of the Internet of Things, which provides the user with a simple, efficient and modern approach to advanced services and information.  

 

Smart speakers use a large set of algorithms that are built into the hardware using the SDK, making it easier to develop and update existing software. Many of the algorithms originally used are not used today anymore, or are only partially used, since AI-based approaches are being increasingly used, because they cope better with the nature and amount of data. Built-in AI algorithms have made a big step in performance due to the processing power of newer and modern chips. Algorithms designed for acoustic processing process acoustic signals at high speed, and allow seamless interaction between the user and the devices in an IoT environment. Algorithms are updated and developed through deep learning, which allows for more functionality and a wider range of device applications.  

 

Natural language generation is an algorithm that uses artificial intelligence to generate reports and other types of text in natural language. NLG uses NLP algorithms and AI to generate summaries of texts and other types of data that are easier for people to understand. Texts generated from NLG systems typically contain words and expressions with well-defined semantics. NLG represents the penultimate step of the successive processes that make up the smart speaker response. There are many algorithms available to generate the language: Markov chains, RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory) architectures, etc.  

 

Smart speakers have a complex configuration in which most of the work is done on the motherboard, or processor core and FPGA. Depending on the speaker or the manufacturer, there are several types of motherboards in use, which ensure greater efficiency and functionality of the device. We also have main memory, LED drivers for capacitive keys, analog-to-digital converters, chips for wireless connection, etc. on the motherboard. Also included is a sequence of microphones that are always turned on, with audio capturing being performed after a specific “keyword” is used.