Scientists fear that the use of AI models like ChatGPT in healthcare will exacerbate inequalities.
Epidemiologists from the Universities of Cambridge and Leicester warn that Large Language Models (LLMs) could exacerbate inequalities for ethnic minorities and low-income countries.
Their concern stems from systemic data biases. AI models used in healthcare are trained using information from websites and scientific literature. However, there is evidence that data on ethnicity are often missing from these sources.
As a result, AI tools can be less accurate for underrepresented groups. This can result in ineffective drug recommendations or racist medical advice.
Find out about our conference presentations
Watch videos of our past lectures for free with TNW All Access →
“It is widely accepted that, for many groups of diseases, there is a different risk associated with belonging to an ethnic minority‘ the researchers said in their thesis.
“If the published literature already contains biases and less precision, it is logical that future AI models will maintain them and tighten them even further.”
The scientists are also concerned about the threat to low- and middle-income countries (LMICs). AI models are mainly developed in wealthier countries, which also dominate the funding of medical research.
Consequently, LMICs are “significantly underrepresented” in health education data. This can lead to AI tools giving bad advice to people in these countries.
Despite these concerns, researchers recognize the benefits that AI can bring to medicine. To mitigate the risks, they propose several measures.
First, they want it Models to clearly describe the data used in their development. They also call for further work on aAddressing health inequalities in research, including better recruitment and collection of ethnicity information.
Training data should be sufficiently representative, while further research on the use of AI for marginalized groups is needed. These interventions, the researchers say, will promote equitable and inclusive healthcare.
“We must be cautious and recognize that we cannot and should not stop progress,” said Dr Mohammad Ali from the University of Leicester.
Comments are closed.