World’s first computer vision-guided brain tumor surgery rescues man’s failing eyesight

A real-time machine intelligence system has supported a neurosurgeon during a live brain operation for the first time, helping doctors remove a pituitary tumor while protecting the patient’s eyesight. The procedure took place at the National Hospital for Neurology and Neurosurgery in London as part of a clinical trial. Surgeons used software developed at University College London to analyze the operation’s live video feed. The system identified critical anatomy as surgeons worked. That gave the team an additional visual reference during one of neurosurgery’s most confined operating environments. For 48-year-old Rhys Hibbert, the technology supported surgery on a brain tumor that had progressively damaged his vision. Doctors successfully removed the growth without harming his eyesight. Reading the surgical field Pituitary surgery presents an unusual challenge for computer vision systems. Surgeons work around structures separated by only tiny distances, including the optic nerves and major blood vessels. A small error can have severe consequences. Damage near the surgical site can cause blindness, stroke, or death. UCL’s system processed the live endoscopic video instead of relying only on scans collected before surgery. It highlighted structures that could present risks as the operation unfolded. Hani Marcus, a consultant neurosurgeon at the hospital and UCL professor, performed the procedure with surgical resident Danyal Khan. The surgeon retained full control throughout the operation. The software served as a real-time support tool, helping the team recognize anatomy that required extra caution. Marcus called the procedure an important first step toward reducing risks during delicate brain operations. Trained on surgical video Researchers developed the system at UCL’s Hawkes Institute and trained it using hundreds of annotated videos from previous endoscopic pituitary operations. That training exposed the software to surgical scenarios accumulated across a large collection of procedures. Sophia Bano, the technical lead for the project, said the system can recognize critical anatomy, surgical instruments and interactions between instruments and tissue. Researchers previously tested the technology as a training tool. The current clinical trial moves it into the operating room for live surgical support. The system runs on NVIDIA’s Clara IGX platform, which targets real-time machine intelligence applications in medical devices. Researchers ultimately want the technology to do more than identify anatomy. Future versions could track surgical instruments and monitor how they interact with tissue. That capability could provide surgeons with additional feedback during complex procedures. Test for real-time surgery Hibbert’s brain tumor measured about 11 millimeters and sat in his pituitary gland. Doctors discovered it after he collapsed during a walk in December 2024. His condition later caused severe hormone problems and worsening vision. He eventually lost much of his peripheral sight and began using walking sticks. His vision improved dramatically after surgery. Within a week, he could walk independently without his glasses or sticks. The trial received funding from the UK’s National Institute for Health and Care Research and Google. The project also received support from UCLH’s Biomedical Research Centre, the Royal College of Surgeons, EPSRC and Wellcome.Get the latest in engineering, tech, space & science - delivered daily to your inbox.Aamir is a seasoned tech journalist with experience at Exhibit Magazine, Republic World, and PR Newswire. With a deep love for all things tech and science, he has spent years decoding the latest innovations and exploring how they shape industries, lifestyles, and the future of humanity.