Putting the “AI” in aiding clinical decisions
How does artificial intelligence “know” what to do?
The computer is given a certain task and practices executing it using various examples. The system analyzes the examples in the course of the training, and on this basis develops a model for processing similar data or tasks in the same way in the future. However, AI does not do all this without any preliminary work – it requires a detailed specification of the scientific problem behind the clinical question, proper high-quality data collection and annotation, and good AI algorithm design and training. The real key to AI is human expertise: Experienced clinical radiologists and technicians annotate millions of images before an algorithm is ready for training. Multiple experts use the mouse to click on the images and define, for instance, what pulmonary emphysema or a lung nodule look like.
In the case of COVID-19, abnormal regions showing airway disease are localized and delineated in the CT image. Once an abnormal region has been detected in an image, the computer can learn from the examples and apply the extracted concepts to new data. The quality of the annotations and data has a major impact on the quality of the final product.
A tracker in a data jungle
Bogdan Georgescu is an expert in AI technologies. Working with an international team, he is developing prototype solutions that will speed up the analysis of radiological patterns in the lungs that could be caused by COVID-19.
What role does CT imaging play in diagnosing COVID-19?
In patients with severe COVID-19, a CT scan will enable the detection of lung abnormalities. The extent of these abnormalities seems to be directly related to the severity of the illness. CT imaging can therefore play a significant role in diagnosing COVID-19 because it gives physicians and scientists valuable insights.
A CT scan of the chest or thorax not only shows the extent of lung disease, but can also reveal how a patient is responding to various therapies and how their recovery is progressing.
Many COVID-19 patients show abnormalities in the lung. The primary features observed on a lung affected by COVID-19 are peripheral focal or multifocal ground glass opacities, consolidation, and crazy-paving patterns. Using special algorithms, signs of the disease can be evaluated quickly and extensively.
An international team at a virtual conference table
The world is currently in a state of emergency, with governments everywhere restricting personal contact. However, this does not prevent researchers worldwide from pioneering new methods and developing new ideas. Siemens Healthineers formed an international and interdisciplinary team comprising AI scientists from the U.S., software developers from India, CT product as well as research and development experts from Germany, and medical experts from around the globe – and brought them together around a virtual conference table, all with a common goal.
.jpg)
Powerful infrastructure
Our “Sherlock” supercomputer runs at 24,000,000,000,000,000 floating operations per second in over 600 experiments a day.
United against the pandemic
In February 2020, SARS-CoV-2 spread like wildfire around the world. In the very early days of the pandemic, Georgescu and his team began investigating how AI could help clinicians fight COVID-19 with faster diagnosis, triaging, and care-pathway planning. The group has leading expertise in developing AI prototype solutions for medical image analysis, and went into battle alongside the clinical experts. Georgescu explains: "Our core competence lies in the provision of AI-based solutions to analyze clinical images – so we put together a team of experts to find out whether and how we can make a contribution with this knowledge."
Team member Thomas Re, an experienced radiologist, remembers the situation very well: “As for everyone, it was an extremely intense moment. Having trained in radiology in Milan, Italy, a location particularly hit by COVID-19 infections and fatalities just before the U.S. outbreak, and having heard from friends and colleagues directly affected, I was particularly aware of the gravity of the situation. I saw my work on the AI COVID-19 project as my best opportunity to provide a solution in future that could help in this time of crisis by working to improve COVID-19 diagnostic tools for the clinicians on the font-line of the pandemic. My task, as a radiologist, was to annotate imaging data for use in “training” the AI system to recognize COVID-19 disease patterns in chest CT data. Pulling together resources from around the globe for this project and to be part of such a team was truly amazing. My contribution would not have been possible without the support of other local and international team members; in particular, that of my radiology colleague Eileen Krieg, MD based in New Jersey, engineer Amit Vaze and his India based team as well as engineer Guillaume Chabin and his data management team in Paris.”
In record time: A prototype in just three weeks
Algorithms must be fed with good examples
Putting the prototype to the acid test
In order to research the effects of COVID-19 on the lung, the AI team needed reliable comparative values as a solid basis for continuing their work. "When you launch a project like this, you need reliable data and advice from clinical experts right from the start. Only then can we develop the actual prototype. This requires effective algorithms and a powerful computer infrastructure. And finally, it must be possible to make the new concept available to several partners in a clinical setting so that they can test its practical suitability," explains Georgescu.
Theory and practice go hand in hand
Therefore, to build, train, and deploy a machine learning model, the AI scientists and engineers worked hand in hand with dedicated clinical partners. “We have managed to engage and effectively work and collaborate across continents to have an efficient data processing pipeline,” says Georgescu.
His team drew on data from COVID-19 cases and additional studies from around 18 collaborating institutes in the U.S., Canada, and Europe – including Foch Hospital (France), Northwell Health (U.S) and Houston Methodist (U.S.), University Hospital Basel (Switzerland), and Vancouver General Hospital (Canada) – as reference values.
The intelligent CT pneumonia analysis2
Any lung abnormalities for which the SARS-CoV-2 virus could be responsible are automatically identified and classified according to their extent. To do so, a quantifying algorithm systematically analyzes a CT image of the patient’s rib cage and looks for opacity, i.e., hardening in the tissue of the lung lobes. If lesions of this type are detected, the algorithm calculates the density with which they occur in the lung. This type of preselection speeds up the clinical process – and every minute saved provides precious time to initiate proper treatment quickly. Finally, a correlation with the patient's actual symptoms is sought, making it increasingly easy to follow the progress of the disease.
What can a quantifying algorithm analyze and depict? A quantifying algorithm analyzes images and looks for opacity to depict the severity and region of the affected tissue – without assigning the findings to the underlying disease.
From crisis comes opportunity
The process that was set in motion by Georgescu and his team could become a model for many future developments in terms of its approach, pace, and guiding principles.
The pandemic is not all that has spread in the last few months: Pioneering spirit, cross-border project work, and intelligent collaborations have also flourished. An algorithm is just one of the many results to have come to fruition – and it has been shown that, in the fight against COVID-19, hard work and creativity are also contagious.
Automated quantification is just the beginning
The prototype was developed to automatically identify and quantify hyperdense lung regions in order to detect the affected area and then assess the severity of the inflamed tissue. This automated quantification of abnormalities associated with COVID-19 from noncontrast chest CT scans could help clinicians to evaluate the disease and to assess its severity and progression. However, the accuracy with which COVID-19 can be distinguished from other types of lung disease on CT images still varies.
One more thing
COVID-19 can manifest similarly to other respiratory infections such as influenza, which can make patient triaging and diagnosis challenging. It is therefore important to be able to distinguish COVID-19-related pulmonary disease not just from healthy lungs, but also from other types of lung diseases, such as other infections, malignancies, interstitial lung disease, and chronic obstructive pulmonary disease.
