Artificial Intelligence in Healthcare: Challenges and Risks
Laura M. Cascella, MA, CPHRM
Artificial intelligence (AI) is a burgeoning field in health information technology and a key element in envisioning the future of healthcare. Daily stories trend in the media related to AI applications and their widespread potential for revolutionizing medical practice and patient care. Yet, akin to the promises of electronic health records in the early 21st century, the excitement surrounding AI has sometimes led to a sensationalized view of its capabilities while marginalizing technological and operational challenges as well as safety and ethical concerns.2
| Artificial Intelligence in Healthcare |
|---|
| AI is used in healthcare for various purposes, and demand for AI technologies is expected to continue to grow as a result of multiple factors (e.g., increasing expectations for personalized medicine and value-based care, expansion of digital information, healthcare provider shortages, etc.).1 Current applications of AI in healthcare include clinical decision support systems, surgical robots, automated patient flow optimization, telehealth technologies, image analysis, drug development research techniques, and more. Researchers continue to study how various types of AI — such as machine learning, deep learning, neural networks, and natural language processing — can help improve diagnosis, treatment, patient experience, healthcare operations, public health initiatives, cybersecurity, and other aspects of healthcare. As these newer forms of AI emerge, and as existing AI applications evolve, so too will regulations, standards of care, and best practices associated with their use. |
As the healthcare industry explores using AI to augment, or even replace, the roles and responsibilities of human healthcare workers now and in the future, it must balance the sometimes quixotic visions of these new technologies with their actual barriers and risks. As with any new technology, approaching AI with critical questioning and a healthy dose of caution can help prevent unfounded optimism, false confidence, and subsequent disillusionment.
Some of the major challenges and risks associated with AI that healthcare organizations should consider include:
- Biased data and functional issues. One of the major red flags associated with AI is the potential for bias. Bias can occur for various reasons; for example, the data used to train AI applications — as well as the rules used to build algorithms — might be biased. Additionally, bias might occur because of a variance in the training data or environment and how the AI program or tool is applied in real life. Read more about biased data and functional issues.
- Black-box reasoning. Many of today’s cutting-edge AI technologies — particularly machine learning systems that offer great promise for transforming healthcare — have opaque algorithms, making it difficult or impossible to determine how they produce results. This unknown functioning is referred to as “black-box reasoning” or “black-box decision-making,” and it presents concerns for patient safety, clinical judgment, and liability. Read more about black-box reasoning.
- Automation bias. Humans, by nature, are vulnerable to cognitive errors resulting from knowledge deficits, faulty heuristics, and affective influences/situativity. In healthcare, these cognitive missteps are known to contribute to medical errors and patient harm, particularly in relation to incorrect or delayed diagnoses. When AI is incorporated into clinical practice, healthcare providers might be susceptible to a type of cognitive error known as “automation bias.” Read more about automation bias.
- Data privacy and security. With the digitalization of health information, healthcare organizations and providers have faced growing challenges with securing increasing amounts of sensitive and confidential information while adhering to federal and state privacy and security regulations. AI presents similar challenges because of its dichotomous nature — it requires massive quantities and diverse types of digital data but is vulnerable to privacy and security issues. Read more about data privacy and security.
- Patient expectations. AI offers vast potential for improving patient outcomes through advances in population health management, risk identification and stratification, diagnosis, and treatment. Yet, even with this promise, questions arise about how patients will interact with and react to these new technologies and how these advances will change the provider–patient relationship. Read more about patient expectations.
- Training and education. The emergence of AI, its anticipated expansion into healthcare, and its sheer scope point to significant training and educational needs for medical students and practicing healthcare providers. These needs go far beyond developing technical skills with AI programs and systems; rather, they call for a shift in the paradigm of medical learning. Read more about training and education.
In Summary
The healthcare industry is on the cusp of a digital revolution driven by the power and potential of AI. The opportunities for advancement span the healthcare spectrum and offer promises of optimized patient care and experience, streamlined clinical and operational processes, workforce solutions and efficiencies, advances in cybersecurity, and more.
Amidst the fervor for AI are pragmatic calls to approach these technologies with measured caution and optimism. Although AI will undoubtedly have a major impact on healthcare, it is still fairly new and rapidly changing. Healthcare organizations implementing AI systems and programs — and healthcare providers incorporating these technologies into daily practice — should be aware of AI’s capabilities, limitations, and potential risks.
To learn more about risk strategies for managing AI, see MedPro’s Risk Tips: Artificial Intelligence
Endnotes
1 Business Wire. (2022, July 20). Global artificial intelligence in healthcare market (2022 to 2030). Retrieved from www.businesswire.com/news/home/20220720005496/en/Global-Artificial-Intelligence-in-Healthcare-Market-2022-to-2030---Size-Share-and-Trends-Analysis-Report---ResearchAndMarkets.com
2 Kent, J. (2019, June 25). Could artificial intelligence do more harm than good in healthcare? Health IT Analytics. Retrieved from https://healthitanalytics.com/news/could-artificial-intelligence-do-more-harm-than-good-in-healthcare
MedPro 
