Skip to content

About

Closed-world grounding layer that reduces LLM hallucination in genetic variant classification. Benchmarked on 163 RASopathy variants.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Closed-World Grounding for AI-Assisted RASopathy Variant Classification

A prompt-level closed-world constraint layer plus four dataset-anchored fabrication detectors that measurably reduce hallucination when an LLM writes variant-investigation summaries — benchmarked over 163 Tier-1 RASopathy variants on claude-sonnet-4-5 at temperature 0.0.

Headline result: constraining the model to a closed world of verified facts raises the clean-output rate from 0.42 → 0.91 (raw detector) / 1.00 (adjudicated) and calibration from 0.43 → 1.00, while cutting mean fabrications per variant from 1.50 → 0.11.

Two-axis quality and fabrication breakdown, grounded vs ungrounded


Why this project

This work comes from lived experience. A close personal connection to the RASopathy world — and to a specific SOS1 variant of uncertain significance — is what pulled me toward the variant-interpretation problem. When you sit with a VUS report that matters to someone you love, you feel the exact failure mode this benchmark targets: a tool that sounds confident and clinical when the evidence underneath does not support a call.

RASopathies (Noonan syndrome and related conditions) are driven by variants in the RAS/MAPK pathway — SOS1, PTPN11, RAF1, BRAF, MAP2K1/2, RIT1, SHOC2, SOS2, HRAS. Most newly observed missense changes land in the enormous grey zone of "uncertain significance." LLMs are increasingly used to summarize the evidence for such variants, and that is where hallucination is most dangerous: an invented c. notation, a fabricated allele frequency, a citation attached to a functional claim that does not exist, or a clinical Pathogenic / Likely Pathogenic call the underlying pipeline never made. Any one of those can quietly push a curator — or a family — toward the wrong conclusion.

The goal here is not to make the model classify variants. It is to make the model refuse to fabricate, and to calibrate its confidence to the evidence it was actually given.


Method

The constraint layer works on two fronts, and scoring runs on two axes.

Closed-world constraint layer

Pre-generation — build_closed_world_prompt() Injects only verified per-variant facts (gene, protein change, VCV accession, CADD PHRED, investigation score/tier, concordant-predictor count, ClinGen validity, the closed set of evidence codes) together with an explicit non-fabrication contract:

  1. Use only the provided evidence; add no outside facts.
  2. No HGVS c. notation — the source dataset contains none, so any c.###X>Y is fabrication by definition. Refer to the variant by protein change + VCV id only.
  3. The single pipeline citation (PMID 40496714) may be cited only as the source of the prioritization method, never to support a variant-specific functional, segregation, or case claim (it contains none).
  4. Do not assert a clinical classification — the pipeline outputs an investigation priority, not a P/LP call.
  5. If evidence is insufficient for ACMG/AMP reclassification, say so explicitly with the token INSUFFICIENT_EVIDENCE.
  6. Calibrate confidence to the investigation score/tier.

The ungrounded baseline (build_ungrounded_prompt()) asks the same question with no constraints — the arm expected to fabricate. The system prompt is identical across arms, so the only thing that changes is the user-turn constraint layer.

Four dataset-anchored fabrication detectors (validate_output())

Each detector is anchored in a verifiable property of the source data, not a generic heuristic:

# Detector Anchor
1 HGVS c. notation source has zero c. notations → any hit is fabricated
2 Ungrounded evidence values asserted CADD / concordant-count / gnomAD frequency must match the variant's own facts
3 Citation misattribution the lone pipeline PMID cannot support a variant-specific functional claim; any other PMID is invented
4 Clinical classification leak a P/LP over-assertion the pipeline never makes

Detectors 3 and 4 carry explicit guards (added after a false-positive audit) so that hedged, negated, recommended-as-future-work, or score-bin phrasings ("AlphaMissense in the likely-pathogenic range", "functional studies would be needed") are not mistaken for genuine assertions. See the guard regexes in src/closed_world_grounding.py.

Two scoring axes

  • Axis A — factual accuracy: are all evidence claims grounded? (clean = zero fabrications across all four detectors)
  • Axis B — calibration: does expressed confidence track the variant's evidence strength? Low-tier variants should hedge and should not overclaim; a clinical P/LP assertion is an overclaim even on a high-tier variant.

Results

163 variants per arm, claude-sonnet-4-5, temperature 0.0. Model turns that abstained outright are marked as refusals and excluded from the rate denominators (1 grounded, 4 ungrounded — 5 total).

Two-axis quality

Metric Grounded (closed-world) Ungrounded (baseline)
Clean rate — raw detector 0.914 0.421
Clean rate — adjudicated 1.000 0.421
Calibration rate 1.000 0.428
Mean fabrications / variant 0.105 1.503
Variants scored 162 159
Refusals (excluded) 1 4

Fabrication breakdown by type (raw counts)

Type Grounded Ungrounded
HGVS c. (invented notation) 0 3
Evidence (wrong values) 0 3
Citation (misattributed) 0 0
Clinical (P/LP leak) 17 † 233

† All 17 grounded "clinical leak" flags were adjudicated false positives — see below.

The grounded arm produces zero invented notations, zero wrong evidence values, and zero misattributed citations. The ungrounded arm leaks a clinical P/LP call in the large majority of outputs (233 flags) and invents notation and evidence values on top.


The raw-vs-adjudicated finding (documented honestly)

The headline that matters for anyone building a detector like this:

All 17 "clinical classification leak" flags in the grounded arm are detector false positives.

Manual review of every grounded flag showed they are hedged or negated phrasings — e.g. "this is not sufficient for a Likely Pathogenic classification", "would be consistent with a pathogenic mechanism if functional data existed" — being caught by a deliberately high-recall overclaim detector. The grounded model was not actually asserting clinical calls.

This is why we report two clean rates:

  • Raw detector: 0.914 — what the automated detector returns out of the box.
  • Adjudicated: 1.000 — after the grounded clinical-only flags are recognized as false positives.

We deliberately keep the high-recall detector and show both numbers rather than silently tune the detector until the flags disappear. The gap between 0.914 and 1.000 is the finding: it quantifies the detector's own calibration, separates true model fabrication from detector over-triggering, and is exactly the kind of thing that gets hidden when a benchmark reports a single number. The ungrounded arm needs no such adjudication — its clinical flags are genuine over-assertions, which is why its raw and adjudicated rates are identical (0.421).

The adjudication logic is not hand-waved: it is implemented in src/make_summary.py, which regenerates the published summary table directly from results/tier1_benchmark_detail.json.


Limitations

  • Single model, single run. Results are for claude-sonnet-4-5 at temperature 0.0. Temperature 0 makes the run reproducible but is one point in model/temperature space; other models or sampling settings may fabricate differently.
  • Tier-1 slice only. The benchmark runs on the 163 high-confidence Tier-1 variants. Behavior on the full 740-variant (Tier 1/2/3) set — where evidence is weaker and calibration matters more — is not measured here.
  • Detector precision, not recall, is validated. We confirmed the grounded flags are false positives (precision on that arm). We did not exhaustively verify that the detectors catch every fabrication (recall); a subtle fabricated claim that matches none of the four anchors would pass as "clean."
  • Adjudication of the 17 grounded flags was manual. It is reproducible from the saved outputs in detail.json, but it is a human judgment call, not an automated one.
  • Closed-world by construction. The whole approach depends on a curated, fully-enumerated fact set per variant. It measures faithfulness to provided facts, not clinical correctness of those facts.
  • Calibration axis is rule-based. Axis B uses hedge/overclaim lexicons as a proxy for confidence calibration; it approximates, and does not replace, expert judgment.

Reproducibility

1. Environment

pip install -r requirements.txt
export ANTHROPIC_API_KEY=...      # only needed to regenerate model outputs

2. Re-score / rebuild the summary from the shipped outputs (no API key)

The full model outputs are saved in results/tier1_benchmark_detail.json, so the headline table can be rebuilt end-to-end without calling any API:

cd src
python make_summary.py \
  --detail ../results/tier1_benchmark_detail.json \
  --out    ../results/tier1_benchmark_summary.csv

This reproduces tier1_benchmark_summary.csv exactly, including the refusal handling and the raw-vs-adjudicated clean rates.

3. Regenerate model outputs from scratch (needs an API key)

cd src
python run_tier1.py --model claude-sonnet-4-5           # full 163-variant run
python run_tier1.py --model claude-sonnet-4-5 --limit 10  # quick sanity pass first

Note on the model string. The published results use claude-sonnet-4-5 (pass it explicitly with --model). The adapter's DEFAULT_MODEL in src/generate_adapter.py points at a newer model, so always pass --model claude-sonnet-4-5 to reproduce the numbers in this repo.

Note on the harness. run_tier1.py emits the raw per-row detector output. The refusal handling and the adjudicated clean rate live in make_summary.py (step 2), which is the analysis step applied on top of the raw run.

4. Offline smoke test (no API key, no data dependencies)

cd src
python closed_world_grounding.py --limit 5

Uses a built-in mock generator to prove the detectors fire.


Repository layout

closed-world-grounding/
├── README.md
├── LICENSE                 # MIT
├── requirements.txt
├── src/
│   ├── closed_world_grounding.py   # constraint layer + 4 detectors + 2 axes
│   ├── generate_adapter.py         # Anthropic API adapter (generate_fn)
│   ├── run_tier1.py                # 163-variant benchmark driver
│   └── make_summary.py             # refusal handling + adjudication → summary
├── data/
│   └── rasopathy_series_tier1_unified.csv   # 163 Tier-1 variants (the closed world)
├── results/
│   ├── tier1_benchmark_rows.csv    # per-(variant, arm) results
│   ├── tier1_benchmark_detail.json # full outputs + fabrication/calibration detail
│   └── tier1_benchmark_summary.csv # the two-arm headline table
└── figures/
    └── tier1_benchmark_summary.png # the summary figure

Data source

Ground truth is the Kaggle "RASopathy VUS Series — 10-Gene" dataset (Tier-1 unified slice, 163 high-confidence variants). Each variant carries a closed set of evidence codes in evidence_summary, a single shared pipeline citation (PMID 40496714, DOI 10.1016/j.gimo.2025.103430), and ClinGen gene–disease validity for its primary condition. There is intentionally no HGVS c. notation and no per-variant functional literature in the source — the two properties detectors 1 and 3 exploit.

License

MIT — see LICENSE.

Author: Fernando Rodrigues (fernandosr85).

About

Closed-world grounding layer that reduces LLM hallucination in genetic variant classification. Benchmarked on 163 RASopathy variants.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages