Healthcare must put safeguards around AI for transparency and security

Four out of 10 patients experience implicit bias in their physicians, according to a MITRE-Harris survey on patient experience. In addition to patients being extra sensitive to provider bias, the use of AI tools and machine learning models has also been shown to be skewed toward racial bias.

On a related note, a A recent study found that 60% of Americans would be uncomfortable with providers relying on AI for their healthcare. But amid provider shortages, shrinking reimbursements and rising patient demands, in time providers may have no choice but to turn to AI tools.

Healthcare IT News WebMD sat down to discuss these concerns surrounding AI in healthcare with Jean-Claude Saghbini, an AI expert and chief technology officer at Lumeris, a value-based care technology and services company — and health provider organization Health IT Leader And what can doctors do about them?

Q. How can healthcare provider organization CIOs and other health IT leaders fight the bias inherent in artificial intelligence as AI systems explode in popularity?

One. When we talk about AI, we often use words like “training” and “machine learning” interchangeably. This is because AI models are primarily trained on human-generated data and thus they learn our human biases. These biases are a significant challenge in AI and they are particularly relevant to healthcare where a patient’s health is at stake and where their presence will continue to propagate healthcare inequality.

To combat this, health IT leaders need to develop a better understanding of the AI ​​models that are embedded in the solutions they are adopting. Perhaps even more important, before they implement any new AI technologies, leaders should make sure that the vendors delivering these solutions appreciate the pitfalls of AI bias and have done their best to avoid it. Developed models and tools.

This can range from ensuring that the upstream training data is unbiased and diverse, or applying transformation methods to the output to compensate for unhelpful biases in the training data.

At Lumaris, for example, we are taking a multi-pronged approach to fighting bias in AI. First, we are actively studying and optimizing health disparities represented in the underlying data as part of our commitment to fairness and equity in health care. This approach involves analyzing health care training data for demographic patterns and adjusting our models to ensure that they do not unfairly influence any specific population group.

Second, we are training our models on more diverse data sets to ensure they are representative of the populations they serve. This includes using more inclusive data sets that represent a wider range of patient demographics, health conditions, and care settings.

Finally, we are embedding non-traditional health features into our models such as social determinants of health data, thereby ensuring predictive models and risk scores account for patients’ unique socioeconomic conditions. For example, when we incorporate SDOH data into AI models, two patients with very similar clinical presentations can be directed toward different interventions for optimal outcomes.

We are also taking a transparent approach to the development and deployment of our AI models, and are incorporating feedback from users and implementing human oversight to ensure that our AI recommendations are in line with clinical best practices. Are.

Fighting implicit bias in AI requires a comprehensive approach that considers the entire AI development lifecycle and cannot be an afterthought. This is really important to promote fairness and equity in healthcare AI.

Q. How do health systems strike a balance between patients who don’t want their physicians to rely on AI and overburdened physicians who are looking to automation for help?

One. Let’s check the first two facts. Fact No. 1 is that in the time between waking up in the morning and seeing each other during an office visit, it is likely that both patient and physician have already used AI several times to ask Alexa about the weather. Asking in, relying on a Nest device for temperature control, Google Maps for optimal directions, and so on. AI is already contributing to many aspects of our lives and it has become inevitable.

Fact No. 2 is what we’re heading towards lacking According to the World Health Organization, by 2030 there will be 10 million physicians worldwide. The use of AI is no longer optional to measure the capabilities of practitioners and mitigate the devastating impact of this shortage.

I completely understand that patients are concerned, and rightfully so. But I encourage us to consider the use of AI in the care of patients, versus patients being “treated” by AI tools, which I believe most people are concerned about.

This scenario has been publicized a lot recently, but the fact of the matter is that AI engines aren’t going to replace doctors anytime soon, and with new technologies like generative AI, we have an opportunity to provide much needed scale. Exciting opportunity. Benefits for both patient and physician. Human expertise and experience remain important components of health care.

Striking a balance between patients not wanting to be treated by AI and clinicians looking to AI systems to help is a delicate issue. Patients may worry that their care is being entrusted to a machine, while physicians may feel overwhelmed by the amount of data they need to review in order to make informed decisions.

The key is education. A lot of headlines in the news and online are created to create havoc and get clicks. By avoiding these misleading articles and focusing on real experiences and use cases of AI in healthcare, patients can see how AI can complement a physician’s knowledge, accelerate access to information, and data. can detect hidden patterns and which can be easily missed. Best of Physicians.

Furthermore, by focusing on facts, not headlines, we can also explain that this tool (and AI is just one tool), if properly integrated into the workflow, is putting the clinician in the driver’s seat. Can enhance a doctor’s ability to provide optimal care. In terms of interaction and responsibility towards the patient. AI is and may continue to be an important tool in healthcare, providing clinicians with insights and recommendations to improve patient outcomes and reduce costs.

I personally believe that the best way to strike a balance between patient and physician AI needs is to ensure that AI is used as a complementary tool for clinical decision making rather than a replacement for human expertise. Is.

For example, Lumaris technology, powered by AI as well as other technologies, is designed to provide physicians with meaningful insights and actionable recommendations that they can use to guide their care while empowering them to make the final call. Can do to guide decisions.

Additionally, we believe it is essential to include patients in the conversation around the development and deployment of AI systems, ensuring that their concerns and preferences are taken into account. Patients may be more willing to accept the use of AI if they understand the benefits it can bring to their care.

Ultimately, it is important to remember that AI is not a silver bullet for healthcare, but rather a tool that can help physicians make better decisions and rapidly enhance and transform healthcare processes, especially GPT. Like with some new basic models.

By ensuring that AI is used appropriately and transparently, and by involving patients in the process, healthcare organizations can strike a balance between patient preferences and the needs of overburdened physicians.

Q. What should provider executives and practitioners be mindful of as more and more AI technologies proliferate?

One. According to the latest AI Index report published by Stanford, the use of AI in health IT is really getting a lot of attention and is a top investment category, but as healthcare leaders we face a dilemma.

Excitement about the possibilities is urging us to move quickly, yet the newness and sometimes black-box nature of technology is raising some alarms and urging us to slow down and play it safe. Success hinges on our ability to strike a balance between accelerating the use and adoption of new AI-based capabilities while ensuring they are implemented with the utmost safety and security.

AI relies on high quality data to provide accurate insights and recommendations. Provider organizations must ensure that the data used to train the AI ​​model is complete, accurate, and representative of the patient population they serve.

They must also be vigilant in monitoring the ongoing quality and integrity of their data to ensure that AI is providing the most accurate and up-to-date information. This also applies to the use of pre-trained large language models, where quality and integrity remain the goal even if the validation approach is new.

As I mentioned, bias in AI can have important consequences in health care, including perpetuating health disparities and reducing the efficacy of clinical decision-making. Provider organizations should be wary of AI models that do not adequately compensate for biases.

As AI becomes more pervasive in healthcare, it is important that provider organizations are transparent about how they are using AI. Additionally, they must ensure that there is human oversight and accountability for the use of AI in patient care to prevent mistakes or errors from being overlooked.

AI raises many ethical considerations in health care, including questions of privacy, data ownership, and informed consent. Provider organizations must be mindful of these ethical considerations and ensure that the use of AI, both directly and indirectly through vendors, is consistent with their ethical principles and values.

AI is here to stay and grow, with new and exciting advances in health care and beyond, especially in generative AI and large language models. It is virtually impossible to stop this growth – and unwise to do so because after decades of rapid technology adoption in healthcare, we have yet to come up with solutions that reduce the burden on physicians while providing better care.

In contrast, most technologies have added new functions and additional functions for providers. With the advent of AI, and more specifically generative AI, we see great opportunities to finally make meaningful progress towards this elusive objective.

Still, for the reasons I’ve listed, we should put up barriers for transparency, partisanship, and security. Interestingly enough, if well thought out, it is these guardrails that will ensure a speedy path to adoption while keeping us from failures that will lead to evolutionary over-reactions towards AI adoption and use.

Follow Bill’s HIT coverage on LinkedIn: Bill Siwicki
Email him at: bsiwicki@himss.org
Healthcare IT News is a HIMSS Media publication.