---

AI in Medicine – Christy

With its rapid development in recent years, artificial intelligence (AI) has gained a huge role in the medical field, undertaking tasks such as medical image analysis, diagnostic support, and patient monitoring. Its transformative power promises advancements in improving our health. However, alongside these benefits, the technology raises ethical concerns and challenges that must be addressed, especially when it is used in environments where vulnerable individuals are affected. This article explores various examples of AI in medicine and its global impact, while also highlighting the ethical dilemmas that the technology may bring.

The kidneys play a vital role in filtering waste and balancing acid within the body. Chronic kidney disease (CKD), also known as kidney failure, can result in the build up of fluid and waste, which often leads to significant complications such as heart disease, anemia and weakened immune system, especially if not diagnosed early. End-stage kidney disease may require dialysis or kidney transplantation for survival, which bring substantial financial and lifestyle burdens. Early diagnosis is crucial, and to combat these challenges, the University of Sheffield and Sheffield Hospitals NHS Foundation’s 3D Lab, kidney doctors and clinical specialists have developed an AI tool to predict kidney failure, six times faster than traditional methods.

The tool was developed by Principal Scientist Jonathan Taylor, who has trained the algorithm and tested it on hundreds of MRI scans. Radiographer Richard Thomas traced round the kidneys in images from a European research study, which then were fed to an AI algorithm that used this data to learn the tracing process, enabling it to carry out the task at high accuracy. Traditional methods were labour-intensive, taking around an hour, as it required manual tracing of up to 60 slices per scan, and by automating kidney volume measurements,  this process can be reduced to less than a minute. The use of this tool allows for timely detection of CKD which could reduce the risk of the disease worsening, and is especially useful in areas that lack medical specialists or diagnostic facilities.

The importance of prompt diagnosis became more evident, when Madhumita Murgia, author of Code Dependent: Living in the Shadow of AI, interviewed Dr. Ashita Singh, the only physician at a hospital in Chinchpada, a rural village in India. Dr. Singh tells the touching story of fourteen-year-old Parvati, who arrived with a chest wound with worsening symptoms. An X-ray was taken to check for tuberculosis, however the results were inconclusive. With no other specialist to ask for a second opinion from, Dr. Singh decided to use the app qTrack, developed by Qure.ai. The app is run by machine learning algorithms trained on past X-rays and sputum test data, designed to spot tuberculosis accurately.

In a race against time and Parvati’s condition deteriorating, qTrack flagged Parvati’s risk of tuberculosis as “presumptive”, allowing her to start treatment immediately instead of waiting for the distant district hospital to confirm the diagnosis. Thanks to Dr. Singh and the assistance of qTrack, Parvati recovered. Since then, Dr. Singh has continued using qTrack with other patients, and regularly gives feedback to Qure.ai. She found that the initial upload process had too many steps, which hindered the efficiency in a hospital where time is limited and lives are at stake. In response, Qure.ai has made the app more seamless, allowing results to be delivered within minutes. This story illustrates AI capability to improve healthcare and shows how technology can adapt quickly, especially to support medical professionals in under-facilitated areas.

Murgia also reveals the racial bias within the medical system, exploring insights from Ziad Obermeyer, a Harvard-trained doctor whose clinical experience spans both in prestigious hospitals of Boston, and the remote Tséhootsoí Medical Centre in the border of New Mexico and Arizona. Obermeyer’s work exposed the medical system’s disparities in the care provided to minority populations. Take the example of the pulse oximeter, a device used to measure oxygen levels in the blood by detecting changes in light absorption of the blood. Obermeyer points out that the device fails to compensate for high amounts of melanin in darker skin, resulting in inaccurate readings. In a 2020 study by the University of Michigan, researchers found that pulse oximeters erroneously assured 12% of Black patients that their oxygen levels were safe, when in reality, they were not. Moreover, Black patients experience almost three times the frequency of undetected occult hypoxemia than white patients. Despite their widespread use, the devices reflect discriminatory design, highlighting how biases are embedded in medical technologies. 

Obermeyer believes that AI systems can serve as a tool in medical decision-making, by addressing limitations of human physicians: correcting cognitive bias, better minority health understanding, and improve predictions of critical health outcomes, with the ultimate aim to close disparities in the system. Nonetheless, he was aware AI systems were far from perfection and inherit deep-rooted biases as they were trained on data chosen by humans. In 2019, Obermeyer analysed an AI system by Optum, a large American healthcare provider, The system was used to recommend extra medical support for people in the United States, through calculating risk scores for each patient. Obermeyer noticed that black patients had inaccurately low scores, as the algorithm was trained to estimate risk scores based on one’s healthcare spending a year, overlooking the fact that not everyone generates cost the same way and how minorities may face barriers when accessing care. On average, healthcare needs of minority populations are higher than white patients. This erroneous design resulted in racial bias where the algorithm was systematically favoritising healthier white patients. Obermeyer helped Optum redesign the model to use data more representative of one’s health.

Similarly, epidemiologist John Lawrance’s 1952 study of osteoarthritis in coal miners in Leigh, Manchester, formed the basis of the “Kellgren & Lawrence’ classification system” which is widely used today to assess the disease. He spent two years comparing their bone structure and blood chemistry with office workers. Obermeyer highlights the flaw in this system: workers in 1950s Leigh were all male and of European ancestry. This narrow dataset limits its applicability to women and other ethnicities. To improve, he developed a software to predict a patient’s pain levels where machine learning algorithms were trained on patients’ self reports, rather than learning from biased doctor assessments This resulted in a far more successful tool that predicted pain, better than human radiologists, underscoring the importance of diverse, representative data in AI to counteract medical biases.

In a high-pressure, life-or-death environment of a hospital, Ashita has noted the crucial responsibility of human oversight when using AI. She has never allowed AI to control her decisions, as its binary output lacks nuance and ethical judgements that complex medical choices require. For example, AI systems such as the “AI Clinician” by Imperial College London show the responsibilities involved in AI medical use. Sepsis, a rapidly progressing and deadly condition, can benefit from AI’s ability to analyse patient data to recommend fluid and vasopressor doses. However, ethical concerns are raised about who is held accountable but the results are inaccurate and patients are affected. Moreover, the lack of transparency in AI systems makes it difficult for medical professionals to trust and use it on vulnerable patients. Human oversight is important, but doctors often need to assess AI suggestions, placing a massive burden on particularly ones who lack the technological expertise to spot errors. Responsibility when AI systems make errors still remains a complex topic with no definite correct solution. 

In summary, AI holds transformative power in the medical field, like improved accuracy in disease prediction and diagnosing, but as demonstrated, it meets significant challenges as biases are deeply embedded in data that are used to train these tools and the complicated ambiguity surrounding responsibility when AI systems fail. Ultimately, the goal should be to harness AI’s abilities to augment, not replace, clinical judgment, especially in an environment where vulnerable lives are at stake, to provide fair and safe healthcare for all.

References

Mayo Clinic – “Chronic kidney disease”: https://www.mayoclinic.org/diseases-conditions/chronic-kidney-disease/symptoms-causes/syc-20354521 (Accessed 10th August 2025)

University of Sheffield – “Artificial Intelligence tool predicts kidney failure six times faster than human expert analysts”: https://sheffield.ac.uk/news/artificial-intelligence-tool-predicts-kidney-failure-six-times-faster-human-expert-analysts (Accessed 10th August 2025)

Madhumita Murgia – Code Dependent: Living in the Shadow of AI (2024), Published by Picador

Michael W. Sjoding – Racial Bias in Pulse Oximetry Measurement”: https://www.nejm.org/doi/full/10.1056/NEJMc2029240 (Accessed 12th August 2025) 

Madeleine Clare Elish and Elizabeth Anne Watkins – “A Study of Integrating AI in Healthcare”: https://datasociety.net/wp-content/uploads/2020/09/Repairing-Innovation-DataSociety-20200930-1.pdf (Accessed 13th August 2025)
Matthieu Komoroski, Leo A Celi, Omar Badawi, Anthony C Gordon, A Aldo Faisal – “The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care”: https://pubmed.ncbi.nlm.nih.gov/30349085/ (Accessed 13th August 2025)