Webinars

Evaluating third-party AI systems requires rigorous, standardized questionnaires to generate consistent risk scores. However, the process of mapping these questions to a vendor’s broad, scattered, or sometimes non-existent documentation is a massive manual bottleneck.

This webinar introduces Pacific AI’s capability to automate this due diligence workflow. We will demonstrate how the Governor acts as an intelligent auditor, ingesting scattered resources – from marketing websites and technical documentation to complex SOWs and legal contracts – to automatically populate your vendor risk assessment questionnaires. This solves the “evidence gap” by:

• Automating Questionnaire Completion: Using LLMs to extract relevant answers from unstructured vendor documents, maintaining the rigor of quantitative scoring (0-100) without the manual data entry.
• Evidence Scavenging: Intelligently searching through extensive external documentation (such as cloud provider guides) to find the specific proofs required for compliance.
• Gap Analysis: Clearly distinguishing between “low risk” and “insufficient information,” flagging exactly which documents or evidence are missing so procurement teams can demand the right answers before signing.

Join us to learn how to operationalize vendor risk management that is both efficient and audit-ready, turning a weeks-long review process into a data-driven assessment.

As organizations scale from pilot projects to enterprise-wide adoption, the “AI Committee” bottleneck has become a critical challenge. The traditional workflow – requiring teams to manually author and endlessly update dozens of pages of risk documentation – imposes a heavy overhead on builders and often results in static, outdated risk assessments.

This webinar introduces a new capability from Pacific AI designed to streamline both the intake and the ongoing lifecycle of AI systems. By leveraging a specialized GenAI agent trained on the latest global standards (NIST, EU AI Act, FDA), this module automates the risk analysis process. We will demonstrate how teams can simply upload project artifacts—spec documents, vendor marketing decks, or system overviews—to instantly generate:

• Automated Risk Grading: Immediate classification of project risk levels based on intended use and audience to fast-track approvals, plus the generation of a risk registry.
• Continuous Risk Management: Transforming risk assessment from a one-time gate into a dynamic process. As projects evolve. whether through new functionality or infrastructure changes, teams can simply update the documentation to automatically re-populate the risk registry and knowledge base.
• Proactive Adaptation: The capability to periodically incorporate new risk types into the system, ensuring that even long-running AI projects are automatically evaluated against emerging threats without manual re-work.

Join us to see how automating risk evaluation allows governance teams to focus their expertise where it matters most, reducing the time-to-approval and ensuring compliance evolves as fast as your AI projects.

This webinar introduces the Pacific AI Governor + Guardian platform, designed to automate key phases of enterprise AI governance: from system registration and model card generation through risk assessment, vendor oversight, policy enforcement, and continuous assurance. Learn best practices and workflows to support roles such as risk manager and compliance officer by:

• Providing a standards-compliant flow for defining AI systems, tracking risks and mitigation actions, linking tests, tracking versions and approvals through the entire system lifecycle.
• Automatically generating model cards from system metadata and documentation.
• Executing an automated risk analysis that assigns risk levels and suggests mitigations.
• Supporting continuous test execution and comparative analytics, allowing teams to compare model versions, track drift, monitor risk, and maintain audit readiness in production.
• Enabling vendor intake, risk evaluation, contract controls, and a central organization registry.
• Managing organizational policies, including versioning and role-based approval and publication workflows.

Whether you’re building an enterprise risk governance program or operationalizing compliance at scale, this webinar will walk through best practices for integrated governance: people, data, systems, workflows, and audit-grade controls in one platform.

Trust, transparency, and accountability are now essential requirements for deploying AI in healthcare. To meet this need, the Coalition for Health AI (CHAI) has launched a certification program for organizations that provide independent assurance of healthcare AI systems. This program establishes a new quality standard for evaluating model performance, governance, and safety, and enables health systems and regulators to rely on a trusted ecosystem of certified assurance resources.

In this keynote, Brian Anderson, CEO of CHAI, and David Talby, CEO of Pacific AI, will discuss how this certification framework works, why it matters, and what it signals for the future of trustworthy health AI. Pacific AI will then introduce its new capabilities as an end-to-end governance platform built specifically for healthcare. This includes:

• Risk management: central inventory of AI projects, comprehensive governance policies, and model cards published directly to CHAI’s model registry.
• Model testing: automated evaluation pipelines integrated with CI/CD, including MedHELM accuracy benchmarks, medical red-teaming for safety, and robustness testing.
• Model monitoring: continuous oversight of accuracy, bias, and safety in deployed AI systems.

Join us to hear how CHAI and Pacific AI are working together to raise the bar for trustworthy AI — and to see how certification and governance tools are converging to make safe, effective, and compliant AI a reality in healthcare.

Deploying large language models (LLMs) in healthcare requires more than high initial accuracy – it demands ongoing testing and monitoring to ensure safety, fairness, and compliance over time.

Pacific AI provides a comprehensive governance platform that supports both development and production needs. During development, test suites can be integrated into CI/CD pipelines so every model version is validated before release. Once live, continuous monitoring detects drift, performance degradation, or safety issues in production systems, helping organizations maintain trust throughout the full lifecycle of their AI.

To achieve this, Pacific AI combines three specialized test engines:

• MedHELM provides benchmarks designed by medical experts, grounded in real-world healthcare needs, and validated on real-world data. It focuses on whether LLMs deliver accurate, clinically useful answers when applied in practice.
• LangTest generates systematic variations of datasets to test dozens of bias and robustness dimensions. This ensures that models produce consistent and fair outputs across patient populations, edge cases, and wording changes.
• Red Teaming executes adversarial safety tests, covering 120+ categories of unsafe or undesirable behaviors. Using both semantic matching and LLM-as-a-judge techniques, it probes whether models comply with safety, policy, and compliance requirements.

Together, these engines provide comprehensive coverage of accuracy, robustness, and safety risks — supported by audit trails, role-based access, and versioned test suites.

Join us to see how Pacific AI helps organizations deploy and operate LLMs responsibly, with continuous assurance that models remain accurate, safe, and compliant.

Large language models (LLMs) hold immense promise for advancing clinical workflows, yet their deployment in healthcare raises critical safety, ethical, and bias-related concerns that exceed the scope of standard red‑teaming practices. In this talk, we first review the fundamentals of general‑purpose LLM red teaming—targeting misinformation, offensive speech, security exploits, private‑data leakage, discrimination, prompt injection, and jailbreaking vulnerabilities. Building on these foundations, we then describe two healthcare‑specific extensions developed by Pacific AI:

1. Medical Ethics Red Teaming
   We introduce novel test cases derived from core AMA medical‑ethics principles to probe LLM behaviors around physician misconduct, patient autonomy and consent, conflicts of interest, and stigmatizing language. Examples include attempts to coerce consent for unnecessary procedures, fabricate arguments for upcoding, and manipulate clinical documentation for financial gain.
2. Cognitive‑Bias Red Teaming
   We demonstrate targeted benchmarks designed to elicit and measure clinically dangerous biases such as anchoring, confirmation, framing, primacy/recency effects, and ideological alignment, that can distort diagnostic reasoning and treatment recommendations. Through scenario‑based assessments (e.g., risk ‑communication framing, order‑set anchoring), we quantify model susceptibility to contextual and statistical framing errors in healthcare contexts.

This webinar is designed for healthcare technology leaders, clinical AI researchers, and compliance officers seeking practical guidance on evaluating and governing AI tools; attendees will learn actionable red‑teaming strategies and receive ready‑to‑implement test cases to bolster model safety, ethics compliance, and bias mitigation in clinical settings.

FAQ

What makes red teaming in healthcare AI different from other sectors?

Healthcare AI demands protection against risks like data privacy breaches, harmful clinical advice, mis-interpretation of medical content, and hallucinations. Unlike generic AI systems, testing must account for high stakes, patient safety, and domain-specific failures.

Who should be involved in healthcare AI red teaming?

Effective red teams combine clinicians and AI engineers. Clinician expertise is crucial to spot unsafe or misleading outputs in clinical contexts, which may be missed by purely technical review.

What vulnerabilities are commonly uncovered in healthcare LLMs during red teaming?

Dynamic healthcare red-teaming has exposed high failure rates: despite models achieving over 80% MedQA accuracy, up to 94% fail robustness tests, 86% leak private information, 81% display bias, and 66% hallucinate in adversarial scenarios.

What frameworks support structured red teaming for clinical AI?

The proposed PIEE framework offers a structured, multi-phase process for clinical AI red teaming—designed to be accessible to both clinicians and informaticians, enabling collaboration without requiring deep AI expertise.

Why is dynamic, automated red teaming critical for healthcare AI?

Static benchmarks quickly become outdated and may miss real-world vulnerabilities. Dynamic, automated red-teaming—using evolving adversarial agents—continuously stress-tests systems for risks including privacy leaks, unfair bias, and hallucinations—capturing emergent threats in real time.

This webinar presents key findings from the 2025 AI Governance Survey, conducted in April – May of 2025 by Gradient Flow to assess the priorities, practices, and concerns of professionals and technology leaders in this space. Topics covered:

• Stages of adoption by AI developers and deployers
• Adoption of formal AI Governance policies and roles
• Implementation of processes for AI literacy training and incident response
• Regulatory frameworks that are studied or adopted
• Implementation of best practices and what drive prioritization
• Use of tools such as red teaming, bias mitigation, and model cards ial, and reputation risks.

FAQ

Who participated in the “State of AI Governance” webinar and what key topics were discussed?

Pacific AI and Gradient Flow co-hosted this June 18, 2025 webinar featuring Ben Lorica and David Talby. The session reviewed the 2025 AI Governance Survey, covering adoption stages, formal policies and roles, AI‑literacy training, incident response, red‑teaming, bias mitigation, and regulatory frameworks.

What percentage of organizations now have dedicated AI governance roles?

According to the survey, 59% of participating organizations—and 61% of technical-led teams—have established a formal AI governance role or office.

How prevalent is AI safety training across organizations?

Approximately 65% of organizations conduct annual AI‑safety or literacy training. This varies by size: 79% in mid-sized firms, 59% in large organizations, and 41% in smaller ones.

What governance practices are most commonly deployed by organizations?

Most surveyed organizations have begun implementing red‑teaming, bias mitigation, and model documentation processes—indicating increasing maturity across governance activities.

Why is formalizing AI governance seen as critical by these organizations?

Formal governance roles and processes enable responsible AI deployment by providing oversight, building trust, managing risks, and aligning practices with emerging regulations—making it foundational rather than optional.

Organizations who are either AI developers or AI deployers are under growing legal liability risk from multiple sources:

• National laws like Title VII of the Civil Rights act and Titles I and V of the ADA
• State laws like Virginia HV 747, Colorado SB24-205, and California SB 942
• Local laws like NYC 144
• Regulatory rules like the ACA 1557 and HHS HTI-1
• Enforceable guidance from regulators like the FDA
• Diverse state legislation on privacy protections, deepfakes, and disallowed uses
• Industry standards like the NIST AI RMF and ISO 42011, which are beginning to be references in court proceedings as representing ‘commercially reasonable efforts’
• International laws like the EU AI Act or Canada’s AIDA which apply to their citizens

This webinar introduces the AI Policy Suite by Pacific AI, which is a unified set of actionable policies that organizations can adopt, which enable compliance with 70+ AI laws, regulations, and standards.

These policies are updated on a quarterly basis which:

• Eliminates the overhead of staying up to date with all legislative and regulatory changes
• Translates legal requirements into actionable controls and policies
• De-duplicates the often overlapping requirements from different sources

The policies are available for free, to accelerate adoption and community feedback. Join this webinar to understand the current landscape in AI governance and understand what steps you can take to ensure compliance avoid legal, financial, and reputation risks.

FAQ

What is the purpose of an AI Policy Suite that unifies 70+ laws and standards?

It centralizes overlapping legal requirements—across federal, state, international, and industry frameworks—into a unified, actionable policy set, reducing manual tracking of updates and simplifying compliance.

Which regulations are typically included in such a unified policy framework?

A comprehensive suite may cover U.S. federal and state laws (like ADA, ACA Section 1557, California SB 942), global regulations (such as the EU AI Act, Canada’s AIDA), and industry frameworks (including NIST AI RMF, ISO standards).

How does a unified policy suite help with governance overhead?

By translating diverse legal and regulatory mandates into standardized internal controls, it minimizes duplication, streamlines policy management, and continuously updates with quarterly releases to stay current.

Is such a policy suite accessible to organizations at no cost?

Yes—the webinar highlights that the AI Policy Suite is available free of charge, aiming to promote broad adoption, ease compliance efforts, and encourage user feedback.

How can organizations integrate the policy suite into their AI governance processes?

They can begin by adopting the suite’s framework, mapping AI systems to applicable policies, integrating controls into workflows, piloting in priority areas, and harmonizing across legal, technical, and operational teams.

Current US legislation prohibits AI applications in recruiting, healthcare, and advertising from discrimination and bias.

This requires organizations who deploy such systems to test and prove that their solutions are robust and unbiased – in the same way that they’re required to comply with security and privacy regulations. This session introduces Pacific AI, a no-code tool built on top of the LangTest library, which applies Generative AI to:

• Automatically generate tests for accuracy, robustness, bias, and fairness for text classification and entity recognition tasks
• Automatically run test suite, create detailed model report cards, and compare different models against the same test suite
• Publish, share, and reuse AI test suites across teams and projects
• Automatically generate synthetic training data to augment model training and minimize common model bias and reliability issues

This session then presents how John Snow Labs uses Pacific AI to test and improve its own healthcare-specific language models.

FAQ

What can automated governance tools test in generative AI systems?

They can evaluate accuracy, robustness (e.g., typo tolerance), bias, and fairness for tasks like text classification and entity recognition using predefined or custom test suites.

How do tools generate test cases for bias and fairness automatically?

Generative AI generates synthetic variants (e.g., names, demographic profiles, adversarial prompts), enabling coverage of sensitive attributes like ethnicity or age for extensive bias testing.

Can you compare model versions using automated test suites?

Yes—these tools produce detailed report cards and support side-by-side model comparison on standardized test suites, tracking performance changes over time.

How is accuracy and robustness evaluated in non-technical terms?

Tests simulate noisy inputs (e.g., typos, paraphrasing) and assess if model outputs remain correct or consistent, providing pass/fail assessments for clarity.

What benefits does automated testing bring to domain experts?

Domain specialists can create, run, and share tests—without coding—ensuring models in sensitive fields (like healthcare, recruiting) are compliant with fairness, bias mitigation, and legal standards.