Imagine a first responder training for an emergency by speaking with an AI assistant. The assistant needs to understand the situation and reply clearly. It also needs to protect sensitive information. How can we tell whether it is suitable for that work?

In our research, we developed EFR–GVA EvalKit, a set of checks for designing and evaluating AI voice assistants for emergency response. It brings practical questions together: Does the assistant respond well? Is it easy to use? Where is the data processed, and does the system meet relevant requirements?

We used the EvalKit in a virtual reality 3D training scenario where Chemical Biological Radiological Nuclear and Explosives (CBRNE) responders practised verbal communication with a generative voice agent (GVA) enabled in a virtual patient (VP) during task urgency. We compared AI GVA assistants running locally on consumer devices with a remote GVA AI assistant, then studied the experience of 19 CBRNE first responders. The findings showed that local assistants can meet responders’ expectations in this setting.

The project offers a practical way to assess these tools before bringing them into emergency training. The interaction exercise with the VP was EFR–GVA EvalKit first test; the same checks can be adapted to other devices and tasks.

A human operator uses a robot’s 3D model to communicate with a Generative Voice Agent (GVA) integrated in a virtual patient (VP) during a toxic-fire 3D simulation. The robot performs basic triage on the GVA-VP before continuing to the hot zone to gather audiovisual evidence. The challenges in the usability of GVAs in applications for emergency first response include privacy and security of data processing, resources usage and latency of AI models inference, and conversational flow with accurate responses.

The results were presented in the EICS26 conference at the Computer Engineering and Informatics Department (CEID) in the campus of the University of Patras, Greece, highlighting the need to observe the unethical practices of big-tech companies and their involvement in human-rights violations:

Francesca Albanese and her UN report: FROM ECONOMY OF OCCUPATION TO ECONOMY OF GENOCIDE https://lnkd.in/d5qiRahX on the presentation to highlight how #BigTech funds and profits from #HumanRightsViolations.

Read the paper in Proceedings of the ACM on Human-Computer Interaction.

View of Patras from the other end of the Rio–Antirrio Bridge