Research
Multimodal Sensing Network System for PERSIST project
PhD student: Valentino Šafran
Mentor: dr. Izidor Mlakar
Research
PERSIST multimodal sensing network
A novel Multimodal Sensing Network (MSN) developed for the EU H2020 project PERSIST can be used to gather patient data directly from patient inputs over smartphones in the form of Questionnaires (e.g. RASA chatbot) and with the help of devices like smartwatches and fitness bands which contain sensors for different measurements related to human health. MSN provides a powerful platform to distribute more information, which must be organized, and also to process it in order to produce most significant information by fusion of multi-data sources. By combining the data from sensors’ subsets, it is possible to infer contextual information, such as: physical activity, location, mood, and social relationships. The MSN system can be seen as an intermediate layer between the sensors (as data sources), and platforms that are storing patient data. Namely, the Open Health Connect (OHC) platform is used as a data storage.
Architecture of the Multimodal Sensing Network System
The architecture of the MSN can be divided into: Apache Camel, Apache ActiveMQ Artemis, and Apache Kafka. Apache Camel is used as external access into the MSN and also as internal connection between Apache ActiveMQ Artemis and Apache Kafka. Apache Camel is running as a Spring Boot application with an embedded Apache Tomcat server, and support HTTP requests. Spring Boot application is running on a virtual machine, while Apache Camel is working like a router that can route data between different protocols. Apache ActiveMQ Artemis is MQTT broker, and is available both internally and externally to mHealth App, which is subscribed to specific topics. REST API is implemented from the Apache Camel side, by using REST DSL with Java and Swagger UI (OpenAPI) that is used as documentation and for testing of the REST endpoints. mHealth App is also communicating with Apache Camel REST API for Questionnaires and user validation. Apache Camel and Apache ActiveMQ Artemis are connected in two different ways. The first way is by using the Java Messaging Service (JMS) that works like MQTT with topics and queues. Apache Kafka is a distributed event streaming platform. In MSN, Kafka is used for handling the data distribution with AI-based microservices.
Project activities
