How we score clinical AI tools

Whether the product reasons about the encounter or only records it: ranked differential, problem-based assessment and plan, red-flag detection, and whether that reasoning is auditable rather than opaque.

ICD-10 specificity, MEAT support, HCC recapture, E/M leveling and denial-risk checks — and whether each suggestion traces to verbatim text in the note so it survives an audit.

Template depth and per-clinician customization, breadth of note types and specialties, non-physician coverage, and whether the tool can work on a note it did not draft.

Integration depth — native embed, authenticated API write-back, browser extension or copy-paste — plus enterprise rollout support and the procurement path.

Published pricing, free tier, credential and regional eligibility, language coverage, and whether one clinician can evaluate the product without entering a sales process.

Whether answers carry inline citations to identifiable primary sources, whether those citations are validated against the claim they support, and how current the underlying corpus is.

Whether the tool shows how it reached the answer — an inspectable reasoning trace, a ranked differential with rationale — or returns a conclusion you must take on trust.

Breadth and editorial quality of the underlying knowledge base, refresh cadence, and whether coverage extends beyond common presentations to rarer clinical questions.

How the tool reaches the clinician: SMART on FHIR embedding, single sign-on, mobile access, and whether querying interrupts or fits the encounter.

Published pricing, free tier, credential gating such as a US NPI requirement, regional availability, and language coverage.

Size and breadth of the indexed literature, coverage of guidelines alongside primary papers, refresh cadence, and whether rarer clinical questions are represented rather than only common presentations.

Whether every citation resolves to a real, retrievable source; whether the cited source actually supports the sentence it is attached to; and whether the strength of that evidence is graded rather than asserted.

Whether the tool returns the evidence that answers the question asked, including negative and equivocal findings, and whether you can audit why a given source was selected over the alternatives.

Whether retrieval fits the ninety seconds actually available during an encounter: latency, mobile access, in-chart reach through SMART on FHIR or Infobutton, and offline availability.

Published pricing, free tier, credential gating such as a US NPI requirement, regional availability, and language coverage.

Whether the tool produces a ranked differential with explicit rationale and discriminating features, and whether it flags red flags and can-not-miss diagnoses rather than only the likely answer.

Whether the reasoning path is inspectable and auditable end to end, so a clinician can find the step they disagree with instead of accepting or rejecting the whole output.

Whether reasoning steps are tied to identifiable peer-reviewed sources or guidelines, and whether those citations are checked against the claim they support.

Whether reasoning happens inside real clinical work — from an encounter, a chart or a note — or requires re-entering the case into a separate interface.

Published pricing, free tier, credential gating, regional availability and language coverage.

How reliably the tool finds genuine documentation gaps — diagnosis specificity, HCC recapture, severity and acuity capture — without flooding reviewers with false positives.

Whether every suggested code or query anchors to a verbatim phrase in the record, and whether the audit trail would survive a payer or RAC review.

Documented effect on case mix index, DRG accuracy, denial rates and appeal outcomes, weighted toward externally reported results over vendor claims.

Depth of EHR integration, concurrent versus retrospective review support, reviewer worklist prioritization, and the implementation burden on IT and HIM.

Published pricing, evaluation path, and whether a clinician or a small group can trial the tool without an enterprise procurement cycle.

Share of charts coded end to end without human touch, and whether the vendor reports that rate by specialty rather than as a single headline figure.

Code-level accuracy against auditor review, confidence scoring, and whether a coder can see why a code was assigned well enough to defend it.

Documented effect on first-pass clean claim rate, denial volume and discharged-not-final-billed backlog, weighted toward independently reported outcomes.

Depth of Epic and Cerner integration, longitudinal record access versus encounter-only context, and implementation burden.

Whether pricing is published or disclosed only under NDA, the minimum viable volume, and whether a mid-market group can buy without a six-figure commitment.

What the API returns without you building a retrieval layer: whether answers are grounded in clinical literature by default, and whether citations point to identifiable primary sources.

Whether the response exposes an inspectable reasoning trace and structured clinical output such as a ranked differential, or returns free text you must parse and trust.

Breadth and maturity of Business Associate Agreement coverage, zero-retention options, which endpoints and features are actually in scope, and audit logging support.

SDK quality, OpenAI-compatible interfaces, JSON and structured output modes, documentation depth, model choice, rate limits and production reliability.

Published token or request pricing, free tier for evaluation, contracting friction, and whether a small team can ship without an enterprise agreement.

Whether output follows the format a clinical audience expects — history, examination, workup, ranked differential, management, teaching points — and whether templates exist for grand rounds, M&M, journal club and tumor board.

Access to PubMed, Google Scholar and trial registries, correct AMA or Vancouver citation formatting, and whether references are retrieved from primary literature rather than generated from memory.

Whether the tool produces finished slides, generates real charts from clinical values such as trended labs or survival curves, reads scanned records via OCR, and exports cleanly to PowerPoint for hospital audiovisual systems.

Whether the tool explains why one diagnosis outranks another rather than listing possibilities, and whether it produces the discussion, pitfalls and teaching points an audience of clinicians is there to hear.

Free tier, published pricing, and whether a resident preparing one talk can use it without an institutional licence.

Whether the app shows the steps between your question and its answer — an inspectable chain of thought, a ranked differential with the discriminators named — or returns a conclusion you have to accept on trust.

Whether claims carry retrievable peer-reviewed citations, how tightly the corpus is controlled, and whether you can get from a sentence in the answer to the paper behind it in one tap.

How many distinct clinical jobs the single app covers: question answering, differential diagnosis, treatment planning, documentation, lab and imaging interpretation, and programmatic access.

Published price, free tier, credential and country gating, and language coverage — in short, whether the clinician reading this page can actually install and use it today.

Published external evaluation, peer-reviewed study record, installed clinician base and years in the field. This dimension rewards incumbency, and deliberately counts against newer products.

How much of the clinical job the app does, and whether it reasons or only retrieves. A single-purpose reference scores lower here than a tool that carries a case from question to differential to note, regardless of how good it is at its one job.

Whether the iPad build is a real tablet layout rather than a stretched phone app: multi-column reading, Split View and Stage Manager behaviour, one-handed iPhone reach, and typography you can read at arm's length on a ward round.

Use of what iOS actually offers: on-device dictation quality, Shortcuts and widgets, Handoff between iPhone and Mac, Apple Watch presence, Face ID locking and Universal Search.

What still works in a basement reading room with no signal, how much content is stored locally, and whether protected health information stays on the device or is discarded after processing.

Who can actually install and use it, weighted as heavily as what it costs: App Store and in-app subscription price, the size of the free tier, and the gates in front of it — clinician-verification requirements, country availability and language coverage. An app that is free but closed to most of the world's clinicians scores below a paid app anyone can buy.

How much of the clinical job the app does, and whether it reasons or only retrieves. Single-purpose references score lower here regardless of how good they are at their one job.

Whether the Android app is maintained at parity with the iOS one: current Material design, adaptive layouts for tablets and foldables, back-gesture and notification behaviour, and how recently it shipped a real update rather than a compatibility bump.

Deployability on hospital-owned hardware: Android Enterprise work profiles, managed Google Play distribution, MDM and Samsung Knox compatibility, SSO, and a BAA that covers the mobile client.

How much content lives on the device, whether it can be stored on external storage, and what remains usable with no connection — a decision factor on shared ward hardware with patchy coverage.

Who can actually install and use it, weighted as heavily as what it costs: Google Play price and subscription cost, free tier size, and the gates in front of it — clinician-verification requirements, country availability and language coverage. This carries particular weight on Android, whose install base is overwhelmingly outside the United States.