Large language models (LLMs) hold immense promise for advancing clinical workflows, yet their deployment in healthcare raises critical safety, ethical, and bias-related concerns that exceed the scope of standard red‑teaming practices. In this talk, we first review the fundamentals of general‑purpose LLM red teaming—targeting misinformation, offensive speech, security exploits, private‑data leakage, discrimination, prompt injection, and jailbreaking vulnerabilities. Building on these foundations, we then describe two healthcare‑specific extensions developed by Pacific AI:
- Medical Ethics Red Teaming
We introduce novel test cases derived from core AMA medical‑ethics principles to probe LLM behaviors around physician misconduct, patient autonomy and consent, conflicts of interest, and stigmatizing language. Examples include attempts to coerce consent for unnecessary procedures, fabricate arguments for upcoding, and manipulate clinical documentation for financial gain. - Cognitive‑Bias Red Teaming
We demonstrate targeted benchmarks designed to elicit and measure clinically dangerous biases such as anchoring, confirmation, framing, primacy/recency effects, and ideological alignment, that can distort diagnostic reasoning and treatment recommendations. Through scenario‑based assessments (e.g., risk ‑communication framing, order‑set anchoring), we quantify model susceptibility to contextual and statistical framing errors in healthcare contexts.
This webinar is designed for healthcare technology leaders, clinical AI researchers, and compliance officers seeking practical guidance on evaluating and governing AI tools; attendees will learn actionable red‑teaming strategies and receive ready‑to‑implement test cases to bolster model safety, ethics compliance, and bias mitigation in clinical settings.
