A4: Integrated state representation

A learned representation is what foundation models produce — a high-dimensional vector space where similar patient states cluster together, learned from training data without explicit human design. The dimensions don't carry pre-assigned semantic meaning; they encode whatever distinctions the training objective rewards. The representation is dense, continuous, and supports gradient-based reasoning naturally.

An ontology-based representation uses a structured vocabulary like SNOMED-CT (~350,000 clinical concepts with defined relationships), the Unified Medical Language System (UMLS), or domain-specific ontologies like the Human Phenotype Ontology or Gene Ontology. Each concept has an explicit definition, type, and relationships to other concepts. The representation is discrete, symbolic, and supports rule-based reasoning naturally.

Learned representations are powerful for prediction but opaque — you cannot easily explain what particular dimensions encode what, which makes regulatory review and clinical audit difficult. Ontology-based representations are interpretable and auditable but rigid — they cannot represent states the ontology designers didn't anticipate, and adding new concepts requires consensus rather than retraining.

For AIHS, the choice matters because regulatory acceptance and clinical trust depend on interpretability, but predictive performance favours learned representations. The likely architecture is hybrid: learned representations anchored to ontological structure (each learned dimension is associated with a small set of ontological concepts it most strongly correlates with), giving the system both the flexibility of learning and the explainability of explicit structure. This is an active research direction with no settled solution.

Most current medical AI handles uncertainty poorly. Standard softmax outputs from neural networks are systematically overconfident, particularly when input distributions shift from training data — a model trained on one hospital's imaging data is often confidently wrong on another hospital's data with subtly different acquisition protocols.

Better methods exist but each has tradeoffs. Conformal prediction provides distribution-free statistical guarantees (e.g., "with 95% probability, the true value is in this interval") but produces wider intervals than ideal, which can reduce clinical utility. Bayesian deep learning (variational inference, MCMC) produces principled uncertainty estimates but is computationally expensive — often 10-100× the inference cost of standard models. Deep ensembles give well-calibrated uncertainty by averaging predictions from multiple independently-trained models but require training and serving multiple models in parallel.

For AIHS, well-calibrated uncertainty is essential because the system must abstain from action when uncertain rather than confidently doing the wrong thing. A model with 95% confidence on a wrong answer in 1% of cases is unsafe; a model with 60% confidence that knows it doesn't know is safer — because the safety architecture (C6) can route uncertain decisions to clinician review.

This is currently a critical gap and a target for Phase I and Phase II research investment. The Phase II decision gate (B3 viability) explicitly considers uncertainty calibration as one of its assessment criteria — without calibrated uncertainty, even mature causal models cannot be safely deployed in autonomous mode.

Continuous learning (B5) updates model parameters as new patient data arrives, but this raises a fundamental regulatory question: a model that updates after FDA approval is not the same model that was approved. Two architectural approaches address this.

Locked-base + adapter: a fixed validated foundation model is paired with a small, version-tracked adapter module (LoRA-style fine-tuning is a common implementation) that updates from new data. The base provides stability and is the object of regulatory approval; the adapter provides freshness and is subject to a separate, lighter-weight change control process. Updates to the adapter are bounded in magnitude so the combined behaviour stays within the validated envelope of the base.

Periodic re-validation: the system accumulates a learning queue of candidate updates without applying them in production. At regular intervals (monthly, quarterly), candidate updates are applied to a staging copy of the system and validated against held-out clinical benchmarks. Only updates that pass validation are promoted to production, with full audit trail of what changed, when, and why.

Both approaches require sophisticated MLOps infrastructure — model versioning at scale, automated benchmark suites that detect drift, rollback mechanisms when updates degrade performance, and lineage tracking so a clinical decision can be traced back to the specific model version that produced it.

The FDA's predetermined change control plan framework provides regulatory guidance for systems intended to update, with EMA and MHRA developing parallel frameworks. The operational details are still being worked out for the AIHS use case. This is largely engineering and policy rather than science — but it is the engineering and policy without which capable models cannot be deployed.