• Skip to main content
  • Skip to secondary menu
  • Skip to primary sidebar
  • Skip to secondary sidebar
  • Skip to footer

How eClinicalWorks Agentic AI Is Helping Revenue Cycle Leaders Improve Financial Performance
  • Opinion
  • Health IT
    • Behavioral Health
    • Care Coordination
    • EMR/EHR
    • Interoperability
    • Patient Engagement
    • Population Health Management
    • Revenue Cycle Management
    • Social Determinants of Health
  • Digital Health
    • AI
    • Blockchain
    • Precision Medicine
    • Telehealth
    • Wearables
  • Life Sciences
  • Investments
  • M&A
  • Value-based Care
    • Accountable Care (ACOs)
    • Medicare Advantage

FDA Awards Cognita Imaging $1.29M to Validate LLM-as-a-Jury Evaluation Across 1 Million Radiology Reports

by Jasmine Pennic 09/16/2026 Leave a Comment

  • LinkedIn
  • Twitter
  • Facebook
  • Email
  • Print

What You Should Know

  • Cognita Imaging, Inc. (a wholly owned subsidiary of Mosaic Clinical Technologies, Inc., operating within the broader Radiology Partners ecosystem) was awarded a $1.29 million research contract by the U.S. Food and Drug Administration (FDA) to pioneer new methodologies for evaluating autonomous, generative AI radiology reports.
  • The 18-month contract, effective June 22, 2026, was funded under the FDA’s Broad Agency Announcement (BAA) program for advanced research and development in regulatory science.
  • The initiative, titled “Virtual Subject Matter Expert Agents for Radiology Report Evaluation,” addresses the clinical limits of traditional human reader studies, which evaluate only a few hundred cases and fail to capture rare edge cases, subtle equipment discrepancies, or regional workflow variations.
  • Cognita will build and validate an “LLMs-as-a-jury” algorithmic framework that runs multiple large language models concurrently to grade and cross-examine both human-written and AI-generated imaging reports, mitigating single-model bias.
  • The evaluation engine will be applied across a massive dataset of approximately 1 million patient exams from a diverse U.S. patient cohort, analyzing performance variance across demographics, care settings, imaging hardware manufacturers, and low-prevalence pathologies.

The “LLMs-as-a-Jury” Evaluation Architecture

The project is led by Principal Investigator Dr. Akshay Chaudhari (Cognita co-founder and Associate Professor of Radiology and Biomedical Data Science at Stanford University) and co-investigator Dr. Louis Blankemeier (Cognita co-founder and CEO):

  • Multi-Model Consensus Scoring: Rather than relying on a single large language model (which inherits that model’s idiosyncratic biases and failure modes) to score diagnostic drafts, the architecture aggregates evaluation outputs across an ensemble of distinct LLMs acting as a virtual panel of expert reviewers.
  • Building on the Open-Source GREEN Benchmark: The system expands upon Cognita’s prior research developing GREEN (Generative Radiologist Evaluation and Error Notation), an open-source evaluation framework designed to capture clinically meaningful discrepancies between ground-truth radiologist reports and AI-generated text.
  • Targeted Radiologist Escalation: The platform isolates high-discrepancy outliers and clinically significant disagreements for expert human review. Radiologists assess whether errors originated from the generative drafting model, discordance within the LLM jury, or ambiguity in the initial human radiologist’s ground truth.

Validation Across 1 Million Real-World Patient Exams

Following core architectural development, the framework will be stress-tested across an enterprise dataset of approximately 1 million patient imaging exams drawn from a diverse U.S. clinical cohort:

  • Edge-Case & Subgroup Discovery: Analyzes model behavior across variable scanner manufacturers, geographic site types, patient demographics, and low-incidence pathologies that are routinely omitted from standard hundreds-of-cases clearance studies.
  • Sample-Size Sensitivity Analysis: Researchers will extract smaller validation sub-cohorts from the 1-million-exam dataset to empirically quantify how many edge-case diagnostic failures are missed when regulatory or institutional evaluations rely on small datasets.
  • Enterprise Radiology Integration: The initiative leverages clinical workflows and data pipelines from MosaicOS and the broader Radiology Partners network, which covers over 4,000 radiologists interpreting 55+ million imaging studies annually.

Regulatory Science Deliverables

Cognita will provide the FDA with open software code, benchmarking guidelines for constructing multi-agent LLM juries, and comparative performance analyses examining premarket clearance and post-market safety monitoring. The contract serves to advance regulatory science standards and does not constitute formal FDA product approval or commercial clearance.


  • LinkedIn
  • Twitter
  • Facebook
  • Email
  • Print

Tagged With: Artificial Intelligence, radiology

Tap Native

Get in-depth healthcare technology analysis and commentary delivered straight to your email weekly

Reader Interactions

Primary Sidebar

Subscribe to HIT Consultant

Latest insightful articles delivered straight to your inbox weekly.

Submit a Tip or Pitch

Featured Insights

How eClinicalWorks Agentic AI Is Helping Revenue Cycle Leaders Improve Financial Performance

Most-Read

Innovaccer-CHIME 2030 Survey Reveals AI Will Automate Nearly Two-Thirds of Administrative Work and Do 3x the Work With the Same Staff

Innovaccer-CHIME 2030 Survey Reveals AI Will Automate Nearly Two-Thirds of Administrative Work and Do 3x the Work With the Same Staff

CB Insights’ 2026 Digital Health 50 Reveals How Agentic AI Is Becoming the Clinical Provider and Medication Access Engine

Oura Acquires Counsel Health for $625M: Scaling Continuous Biometrics into On-Demand Medical Care

M&A: Oura Acquires Counsel Health for $625M: Scaling Continuous Biometrics into On-Demand Medical Care

WittKieffer 2026 Healthcare CIO Survey: 82% Sit on Executive Teams, Yet Only 36% Have Formal Succession Plans

WittKieffer 2026 Healthcare CIO Survey: 82% Sit on Executive Teams, Yet Only 36% Have Formal Succession Plans

CVS Rebrands Health Services Segment, Launches Model for Retail Pharmacy Reimbursement

CVS Health Formalizes Tax-Free Split-Off to Carve Out CVS Pharmacy by 2028

SAMHSA and ONC Invest $20M in Behavioral Health IT Initiative

ONC Finalizes HTI-5 Deregulatory Rule to Cut $1.55B in Health IT Compliance and Accelerate FHIR APIs

Redesign Health Merges with Sheba Medical Center’s ARC Global and Agrees to Acquire AVIA

Redesign Health Merges with Sheba Medical Center’s ARC Global and Agrees to Acquire AVIA

Sword Acquires Headspace in All-Cash Deal Worth Up to $300M to Unify Physical and Mental AI Care

M&A: Sword Acquires Headspace in All-Cash Deal Worth Up to $300M to Unify Physical and Mental AI Care

M&A: Francisco Partners to Acquire Weave in $650M Take-Private Deal

M&A: Francisco Partners to Acquire Weave in $650M Take-Private Deal

Epic Launches One-Click Care Everywhere Diagnostic Image Exchange

Epic Launches One-Click Care Everywhere Diagnostic Image Exchange

Secondary Sidebar

Footer

Company

  • About Us
  • 2026 Editorial Calendar
  • Advertise with Us
  • Reprints and Permissions
  • Op-Ed Submission Guidelines
  • Contact
  • Subscribe

Editorial Coverage

  • Opinion
  • Health IT
    • Care Coordination
    • EMR/EHR
    • Interoperability
    • Population Health Management
    • Revenue Cycle Management
  • Digital Health
    • Artificial Intelligence
    • Blockchain Tech
    • Precision Medicine
    • Telehealth
    • Wearables
  • Startups
  • Value-Based Care
    • Accountable Care
    • Medicare Advantage

Connect

Subscribe to HIT Consultant Media

Latest insightful articles delivered straight to your inbox weekly

Copyright © 2026. HIT Consultant Media. All Rights Reserved. Privacy Policy |