Roberto Rodriguez Rubio, MD may be a neurosurgeon, but in his work as director and principal investigator of the Skull Base and Cerebrovascular Laboratory, he explores techniques borrowed from photography, 3D art, virtual reality, and even archeology. He is passionate about combining these technologies with cadaveric simulation models in novel ways with the aim of broadening access to neuroanatomy education and improving results in the operating room.
Before joining the UCSF faculty in 2017, Dr. Rodriguez Rubio received his MD from Universidad de Guadalajara in Mexico and completed residencies in General Surgery and Neurosurgery at Antiguo Hospital Civil de Guadalajara "Fray Antonio Alcalde" in Guadalajara. He then came to UCSF as an international observer, falling in love with the research and clinical environment and ultimately pursuing a Skull Base and Cerebrovascular Research fellowship. Upon completion of his fellowship and by recommendation of former lab members, Dr. Rodriguez Rubio assumed the position of lab director with the aim of pushing the boundaries of neurosurgical anatomy education and research at UCSF.
Dr. Rodriguez Rubio using a 3D printer to create neuroanatomical models from real specimens.
Why study neuroanatomy?
There is a huge need to understand neuroanatomy as seen in the lab. We can perform safe procedures knowing general surgical anatomy. But when pushing the boundaries to improve procedures, it is critical to understand the architecture of detailed anatomical structures and their functional implications. Surgeons in other specialties might know every landmark they encounter during surgery, but the topography of the central nervous system is still largely unknown. The anatomy is often more complex than the surgical technique, and often the available references are not accurate enough.
I see technology as the stepping stone to advancing surgical techniques. Just as the introduction of tools like the microscope and endoscope allowed us to visualize and classify anatomy in new ways, our lab integrates new technologies with the aim of improving the free knowledge of anatomy and ultimately offering less invasive surgeries and better clinical outcomes. We use techniques from professional photography, industrial 3D scanning, 3D sculpting, 3D printing, extended reality (an umbrella term for mixed reality, including virtual reality and augmented reality) and Artificial Intelligence (AI) for computer graphics. Everything originated from my passion not only to understand the intricacies of surgical neuroanatomy, but also to portray anatomy in new ways and in 3D.
How do 3D techniques advance our understanding of surgical neuroanatomy?
As neurosurgeons, it is difficult to conceive the real scale of structures. Take white matter tracts: we usually see them on an MRI, but what is the actual size of the structure? That is something you want to know when operating. We know from the literature that perceiving depth helps us understand anatomy. I was shocked when I found out that neurosurgeons are still using stereoscopy, which is a method to create depth perception that goes back to the time of Da Vinci. The two images used in stereoscopy are not enough to understand the multi-layered structures part of a novel surgical approach.
I wondered: what if instead of two photos, we take 200? That is how I found photogrammetry. Photogrammetry is a technique primarily used in the VFX and game industries that involves taking multiple photos from different angles, like an array or scan, and then using software to take measurements from the photos. Right now, photogrammetry is an essential part of our pipeline for going from a cadaveric dissection in the lab to 3D printed or virtual models. Another technique we use is structured light scanning. It has a higher accuracy than photogrammetry, which becomes relevant when reconstructing submillimetric vessels or nerves. It involves using a specialized blue light scanning device that captures how light hits the surface of the object and then reconstructs a 3D representation.
Lately we have also been using neural networks known as instant NerF, which is software that uses AI to reconstruct 2D images into complex 3D scenes with realistic interaction between illumination and anatomical surfaces. A year ago, it used to take days—now it takes seconds.
There are also certain techniques from the photography world that have not been explored in anatomy. UV Fluorescence photography enables visualizing enhanced textures of bones, muscles, vessels and even the subcortical structures of the brain. Full-spectrum photography (i.e., UV reflectance and Infrared) and macrophotography techniques help capture contrasts, and details of the neuroanatomy that we cannot see with the naked eye. These approaches allow us to ask new questions and better understand microsurgical and endoscopic anatomy.
Are there challenges that go along with such an interdisciplinary approach?
These techniques require not only strong skills in anatomy, but also 3D art. We do collaborate with 3D artists and engineering students, but most of the time I am trying to do both to the best of my knowledge. I have been learning a lot from online resources and young students that are very into 3D software and computer graphics.
The demand for 3D work has increased, but for the medical world, the transition has been fairly recent. We were the first medical group working to publish about these 3D models, and it was a difficult process because the concept and technology was so unfamiliar to neurosurgical journals. Fortunately, after some time, we were able to create the first collection of PubMed-indexed 3D articles (i.e., UCSF Surgical Neuroanatomy Collection) that offer a comprehensive review of the main neurosurgical techniques using 3D models and stereoscopic media.
The upside is that we are connected to people beyond the neurosurgical world. I was working on a project with a 3D analyst from UCSD who connected me to a technology used by archaeologists involving laser scans similar to structured light scanning. This ‘Point Cloud’ method enables rendering several large-size assets integrated in a single online viewer. I wondered if we could try this with the Visible Human data sets, which is a multimillion dollar project sponsored by the National Library of Medicine with over 100 gigabytes of beautiful cryosection photos of full-body donors, and that is what we did. With the workflow we developed and the online viewer from this project, we got to the resolution of single neurons. Eventually we will see the democratization of molecular reconstructions, with multi-resolution, multi-scale representation of anatomy down to microscopic structures all in an online viewer.
In this lab, it is not about discovering things that have not existed. Archaeologists are doing this with 3D: What can we do with it?
What do you think is the broader impact of this work?
Seeing the students, residents, and fellows and how they perform after being here is rewarding because I see the results in the short term. They have a whole different way of thinking, not only about neuroanatomy and research, but also how to integrate 3D technologies.
There is an additional layer of knowledge that comes from exploring anatomy in a 3D space. I want to share that experience more broadly. We share our 3D reconstructions and techniques on open-source software in open neurosurgical journals so that it is free to access and can be replicated. With photogrammetry, for example, anyone with a camera can try that technique.
Reconstructions are also a way to virtually plastinate the specimens: after we return the donor, I can go back to the virtual representation and continue to learn from them and the anatomic variations we found and use the 3D assets to teach residents in a more engaging and interactive way. I like to treat the donors as if they were still patients. They decided to donate their bodies so we could discover something new. To honor that, the fellows will spend weeks or months planning the dissections involved in their research projects, like they were planning a surgery, so they can try something meaningful. These donations are a precious resource for learning, and we treat them with respect. Not just because of the logistical difficulty of finding donors, but because they were people and they are our teachers in the lab. The 3D reconstructions are a way to preserve them and to honor their wishes, now in the virtual space.
Interview August 2022. Edited for brevity by Maritte O'Gallagher.
