Published in The Breast, the technical pilot study describes the development and feasibility of an augmented reality-guided system for non-invasive tumour localisation. The work is the result of several years of research led by Tiago Marques, Pedro Gouveia and João Santinha, Group Leaders of the Digital Surgery Lab (DSL) together with several team members such as the PhD student Rafaela Timóteo who helped on the project development as well as data acquisition and analysis.
One of the main challenges in breast-conserving surgery is accurately identifying the tumour's location while preserving as much healthy tissue as possible. To address this, the team developed a system that combines patient-specific 3D anatomical models, known as digital twins, with augmented reality visualisation. The technology enables surgeons to see digital information aligned with the patient's anatomy in real time, without the need for invasive tumour-marking procedures.
Developing the system required close collaboration between engineers, researchers and clinicians. Working alongside surgeons from the Clinical Champalimaud Centre, the team adopted a co-design approach to create an interface that improves spatial understanding of tumour location while fitting naturally into the surgical workflow. The platform automatically aligns each patient's digital twin using external sensors, allowing the virtual model to be accurately registered to the patient's body.
“During my PhD, I wanted to tackle three current challenges in augmented reality–guided breast cancer surgery: accurate patient detection, digital twins registration in the operating room and the development of an intuitive surgeon-centered user interface to enable accurate perception of the digital contents. The ultimate objective was to achieve accurate, non-invasive tumor localization,” explained Rafaela Timóteo, PhD student at the DSL.
The technology was evaluated in four breast-conserving surgeries, where it demonstrated promising tumour localisation accuracy, high usability and seamless integration into existing clinical workflows.
For Tiago Marques, co-group leader of the DSL and one of the study supervisors, the work represents an important step toward the future of image-guided surgery. “By bringing advanced digital technologies directly into the operating room, we can support clinical decision-making, reduce the likelihood of re-excisions, and, most importantly, improve the quality of life of breast cancer patients.” he said.
“Reaching this stage required a truly interdisciplinary effort,” said Pedro Gouveia, Chief Medical Officer, co-group leader of the DSL and also supervisor of this project. “This work combines medical imaging, artificial intelligence, computer vision, extended reality and human-computer interaction, bringing these fields together in direct support of surgical care.”
The project was developed within a strong collaborative ecosystem involving the Breast Cancer Research Program (BCRP), whose commitment to clinical excellence and medical innovation creates an ideal environment for translational research. It also benefited from the expertise of the IT People Innovation team, whose software developers worked alongside DSL researchers to transform complex algorithms into robust and clinically relevant tools.
The work was supported by the Portuguese Recovery and Resilience Plan (PRR) and the Health from Portugal consortium, initiatives that are helping accelerate the digital transformation of healthcare in Portugal.
While further validation is needed, the study highlights the potential of patient-specific digital twins to become a valuable surgical tool helping clinicians visualise anatomy more clearly, plan procedures more effectively and perform surgery with greater precision.
Original paper here.
Text by Andreia Pinho, Communication and Events Officer at the Champalimaud Foundation's Communication, Events & Outreach Team
