Author: Žiga Pustoslemšek
Mentor: Matej Rojc
Comentor: Izidor Mlakar
Degree: Study program Electrical engineering, course electronics
Date: September, 2019
DKUM: *
The diploma thesis presents the development of two independent applications that use the Microsoft Azure Face service for face recognition. The first application is designed in order to manage the users that we want to include in the face recognition system. Another application was developed for the process of identifying persons in the IoT environment. In the thesis are presented methods for face detection and recognition, as well as the technologies used to develop the system and both applications.
The aim of the thesis is to develop a comprehensive system for the detection and recognition of faces of persons in IoT environments. The proposed solution design follows the modularity and paradigm of distributed processing. Key, system-dependent and processor-intensive elements, e.g. image processing, detection and facial recognition are performed remotely on the Azure Face API cloud platform. In the local environment (i.e., on the edge), processor less demanding services are implemented, e.g. image capture, RESTFUL communication between edge and cloud components by implementing appropriate programming interfaces, i.e. APIs (Application for recognizing persons). As part of a comprehensive solution, we have set up a web-based user management application that directly communicates with the Microsoft Azure cloud platform through RESTFUL communication and associated APIs. The application consists of two modules: a registration and user management module, and a module for adding new faces to “registered” users. We will also integrate a comprehensive face detection and recognition system with the Home Assistant platform. Within the Digital Signal Processing Lab, the Home Assistant platform is the focal point of the IoT environment.
We divide the whole recognition process into several processes. In order to successfully identify a person, the processes in the system must be performed in the following order: camera image capture, detection, identification, acquisition of relevant person data, and forwarding them to the Home Assistant platform.
.
In the first process, we get an image from an IP camera. We connect to the camera using the python package CV2. The IP address, username, and password must be pre-configured to connect. When this step is done, the system can connect to the camera and capture the video signal. The captured video signal is made up of many pictures, respectively, about 30 frames per second. Only one of the images is then used in the following process, which is intended for face detection.
Furthermore, we need to perform face detection on the captured image. The image must therefore be transcoded into binary format. In this form, it is then forwarded to the Azure cloud platform, where a face detection algorithm is used. In case of successful face detection in the transmitted image, we also receive a response from the Azure cloud platform, which provides information about the detected faces. The transmitted data are unreadable and are only useful for visual display, but in the case of the proposed system they are used for further processing (face recognition).
The third step is to identify the persons in the captured image. In doing so, we take advantage of the data obtained in the previous step, namely the “faceId” data – which is a unique code that identifies facial features. All face identification codes must then be stored in the box and sent to the Azure Face Recognition API. In addition, we send a group variable, by which the algorithm compares the data obtained with the stored faces on the Azure platform. In the case of recognizing persons in any of the defined groups, we receive a response from the Azure cloud platform, which provides information about the candidates and the confidence of the algorithm for correctly recognizing the person in the image.
In the previous step, we identified the persons. In case of successful identification, the received response from the Azure cloud platform contains the information. Due to the need for more specific information from the identified person (name, description), we have to repeat the call to the Azure cloud platform in the proposed system and use the API to get the specific person. The problem is that the system would use up the limited amount of API feature calls available on the Azure cloud platform at a rate of about 30,000 calls per month fairly quickly. For this reason, we minimized the performance of such calls to obtain more detailed information about persons by checking each time a person is identified for memory. In the absence of such records, we called the API function to retrieve all group members again, and then stored them in memory. In this way, we significantly reduced the number of calls.
In the last step, in the proposed system, all the acquired data must be forwarded to the IoT platform of the environment with an IP camera, which in our case is the Home Assistant platform.
In addition to the facial recognition system, we also need an Azure cloud management application to add new faces, people, groups, and more. Due to the nature of the Azur system that works through API calls (which is not exactly the most user friendly way), we have developed a web application to allow users to access and manage the system more easily. We used the MERN environment to develop the web application.
.
Development of an advanced system for lane detection on GPU platforms
