The purpose of this paper is to provide a growth path to AI development. It is based on principles learned from launching CT, MRI, Neuro Vascular, and PACS.
- Create a model (The Knowledge Model) that will build with additional concepts as they develop, resulting in synergistic thinking (Causal Knowledge by Judea Pearl – one of the Fathers of Artificial Intelligence).
- Select a leading factor (Intuition) that will be tested for its leadership as the process continues. Crucial for medicine.
- Select a leading process (Clinical/Technical Tie) that is well proven. Created by Dr Alex Margulis, UCSF Chair when CT was being established.
- Manage technologies (2D/2.5D/3D/Holography etc.)
- Build synergies (e.g. radiology/surgery) to expand interoperability.
An Opportunity – Colonoscopy in True3D Space
Let’s begin with an example of colonoscopy using interactive mixed reality (IMR). Think about a hologram where the user can “swim” through the object (True 3D). The goal is to increase patient outcomes when compared to optical colonoscopy (present standard) and virtual colonoscopy (with visualisation limited to 2D views). Optical colonoscopy misses about 40% of flat lesion cancers (cancers that are on the inner wall of the colon and are therefore very challenging to visualise.) 2D virtual colonoscopy misses between 20-80% of flat lesion cancers. True 3D colonoscopy has a flat lesion detection close to 100% (UCSF/Clinical Trial) (Table 1) (Yee 2016).
2D Views vs True 3D Visualisation
Let’s examine how experts in the field of medical imaging describe 2D views vs True 3D visualisation for brain and spine imaging and the impact on intuition. Meaningful data and insights are embedded within medical images. Often insights are undetectable via routine visual analysis with 2D/2.5D view (2.5D views represent a 2D view of a 3D volume object.) Valuable information is being overlooked and the optimum level of intuition is not realised with 2D views. It is important to understand that images are more than pictures - they are data (Gillies et al. 2016).
When a physician examines a CT, they’re piecing together multiple 2D perspectives—or 3D perspectives on a 2D screen—to imagine a patient’s 3D anatomy. That mental leap means they’re forced to make assumptions about what the patient’s colon truly looks like—which can slow down workflow and open the door to overlooking critical clinical information. True 3D uses existing medical image datasets to give physicians an interactive, three-dimensional solution that may make reading medical images more intuitive, help physicians reach a diagnosis, and assist in complex surgical planning applications.
For brain, 3D is the answer. “It is no longer adequate to analyse complex brain and skull-base tumours with 2D views….. the future of neuroradiology is in advanced techniques such as virtual reality and augmented reality, which produce a dynamic, interactive 3D view of the patient’s imaging” (Kumar 2018).
For spine, again 3D is the answer. Scoliosis is a 3D problem. IMR could be utilised for the surgeon to accurately position the placement of rods to achieve the best efficacy for treatment – e.g. addressing the twist in the spine. IMR technology takes treatment from “best guess” to “I know intuitively I am correct.” IMR is used by spine, gastro and pulmonary experts to confirm the ideal solution (Ross and Burnett 2018).
Cheryl Petersilge, former CMO at the Cleveland Clinic, stated that, “One of the critical shifts that we as radiologists, or in the radiology industry, need to think about, is shifting our focus to the outcomes at the enterprise level, and use our radiology environment and enterprise imaging as tools (e.g. imaging in open 3D space), to affect the enterprise outcomes” (Petersilge 2019).
The Knowledge Model and Enabling AI
Let’s look at some definitions that will be helpful in analysing the Knowledge Model and enabling Artificial Intelligence (AI). Knowledge is composed of cognition (vertical axis of the model) and intuition (horizontal axis of the model) (Figure 1).
Knowledge drives clinical efficacy (CE) + workflow (WF) = patient outcomes (PO). Intuition is more powerful than cognition and uses human intelligence (HI). Intuition (HI) drives “Doing the Right Things.” Cognition uses Machine Intelligence (MI); Big Data; AI; etc. Cognition drives “Doing Things Right.” Protocols (clinical/technical tie) is driven by HI. The Intuition Engine provides the required structure to minimise cognitive bias and maximise the intuitive process.
AI provides significant potential for improving both knowledge and patient outcomes. To achieve these powerful results, there are several steps that are proposed for users to follow:
- The Clinical/Technical Tie – having Clinical and Technical team members working closely to ensure “Doing The Right Things” (DRT) by use of Human Intelligence.
- Step (1) will lead to “Protocol Development” that delineates paths to follow for ”Doing Things Right” (DTR).
- DTR will result in sections of the protocol that allow for creation of AI steps – e.g. partitioning a long object into sections, or detection of abnormal tissue shapes, or characterising the specific tissue as cancerous.
The Key Ingredient – The Protocol
- The Clinical/Technical Tie – Physician/Scientist – deep understanding using human intelligence.
- Thinking “outside the box” – driven by intuition.
- Dreaming in open 3D space, “creating a new box.”
- For True 3D Virtual Colonoscopy – dividing the colon into linear sections, studying one section at a time to find polyps = a new way of thinking.
These results can’t be created by studying an infinite array of 2D/2.5D images. They have been created by human intelligence at work in an environment established by True 3D-Open 3D space imaging.
Let’s focus on the AI steps being selected. They would not exist in analysing the colon without true 3D imaging. That is, without the protocol that was selected.
Protocol Example – Left Atrial Appendage
Patients with atrial fibrillation can be seriously harmed by clots coming out of the left atrial appendage (LAA). An implant device can block the opening to the LAA. The goal is to accurately size the implant for breadth and depth of placement and guide it into the ostium.
The solution is to get to the face of the ostium and measure for implant size. With interactive mixed reality, it takes approximately 1.5 minutes to reach the target and click. With 2D/2.5D, it takes 20 minutes.
For virtual colonoscopy, intuition is the key for developing the protocol process. For the LAA, intuition plays a key role in levelling the playing field in dealing with patients of various sizes and shapes. Interactive mixed reality provides the intuitive capability.
The Big Win - The Intersection of Big Data and IMR
IMR drives the intuitive element of knowledge while Big Data drives the cognitive element of knowledge thus reducing cognitive bias. The potential for a big win would be:
- IMR – initial protocol for increasing patient outcomes (Doing the Right Things)
- Big Data - using 1. provides best practices to optimise the IMR Protocol (Doing Things Right)
The Future Path
Big Data/ML are not sentient beings. They are not able to perceive and feel things. AI must be embedded into the complex workflows of healthcare to significantly create value. This is exactly the perspective that was followed in this paper. The intuitive aspect of imaging allows doctors and scientists to work together, in open 3D space to create protocols focused on patient outcomes, i.e. DOING THE RIGHT THINGS. The cognitive aspect of imaging then provides for optimising the protocol and patient outcomes, i.e. DOING THINGS RIGHT. The Knowledge Model, characterising the cognitive/intuitive activity represents the overall interaction. It is becoming evident that when protocols are jointly developed (e.g. radiology and surgery), interoperability and enterprise are complete with improved patient outcomes.
Dr Lenny Berliner, Interventional Radiologist sums it up well when he says, “Interactive Mixed Reality provides a direct path to intuition. The viewing of 2D/2.5D views actually creates an impediment to intuition”.
Conflict of Interest
There are no conflicts with regard to any of the material provided.
Sergio Aguirre, EchoPixel Chairman and CEO for creating the future, Dr Frandics Chan, Stanford University Radiology as key clinical developer, Dr Judy Yee, Chair Montefiore Hospital for providing clinical guidance for the colonoscopy programme at UCSF, Alex Seelenberger for being an outstanding entrepreneur by being the first investor and Dr Thomas Fogarty for his insight and guidance and support.