Can a bot take over primary care?

Babylon boasts that its artificial intelligence bot performs better than human doctors at a medical exam. Not surprisingly, such claims have elicited doubts and protests from practitioners and medical groups. 

For starters, the Royal College of General Practitioners contends that no app or algorithm will be able to do what a GP does. 

"An app might be able to pass an automated clinical knowledge test – but the answer to a clinical scenario isn't always cut and dried," the group explained. "There are many factors to take into account, a great deal of risk to manage, and the emotional impact a diagnosis might have on a patient to consider."

"There are many factors to take into account, a great deal of risk to manage, and the emotional impact a diagnosis might have on a patient to consider."

The chatbot AI has been tested on what its creators said was a representative set of questions from the Membership of the Royal College of General Practitioners exam. The MRCGP is the final test set for trainee GPs to be accredited by the organisation. 

The first time the AI bot took the exam, Babylon said, it got a score of 81 percent. In contrast, the company noted the average mark for human doctors was 72 percent, based on results logged between 2012 and 2017.

However, the RCGP raised doubts about the test's questions because, as Prof. Martin Marshall, one of the RCGP's vice-chairs, pointed out: "The college examination questions that we actually use aren't available in the public domain."

Babylon said it had used example questions published directly by the college and that some had indeed been made publicly available.

"We would be delighted if they could formally share with us their examination papers so I could replicate the exam exactly. That would be great," Babylon chief executive Ali Parsa told the BBC.

The AI bot was unveiled at an event held at the Royal College of Physicians. NHS England chairman Sir Malcolm Grant, who attended the event, appeared to be more receptive. 

"It is difficult to imagine the historical model of a general practitioner, which is after all the foundation stone of the NHS and medicine, not evolving," he said. "We are at a tipping point of how we provide care."

To further test the AI, Babylon partnered with doctors at two U.S. organisations – Stanford Primary Care and Yale New Haven Health – as well as doctors from the Royal College of Physicians.

Babylon said they had developed 100 real-life scenarios to test the AI. The company added that it expected its chatbot's diagnostic skills would further improve as a consequence. The AI makes a diagnosis by offering several possible scenarios along with a percentage-based estimate of each one being correct.

Mr. Parsa disputed the idea that doctors would be left out in the cold, explaining that the intention was still for a medic to follow up the AI's diagnoses.

"We are fully aware that an artificial intelligence on its own cannot look after a patient. And that is why we complement it with physicians," he said. "It is never going to replace a doctor, but just to help."

Source: BBC News