BE Health Ventures | VP of Investment & Clinical Strategy | Hsin-Wei Shen
For AI SaMD (AI-based software as a medical device) teams, getting cleared and passing IEC 62304 is rarely blocked by insufficient model accuracy. The real bottleneck is that teams try to force AI uncertainty into a lifecycle framework designed for deterministic software—then collapse under “complete-looking” documentation that is impossible to maintain. The only strategy that truly works is this: wrap AI uncertainty inside a deterministic governance framework, and build a minimum-but-complete design-control and traceability structure that regulators can understand, while your team can still manage cost and development pace.
1. First, Define What AI Is: It’s Not an Exception, It’s a Software Item
IEC 62304 does not change the rules because you use AI. It cares about two things: software safety classification (A, B, C) and lifecycle activities. Therefore, in the very first section of your documentation, you must place the AI module back into IEC 62304 language: the AI module is typically treated as a Software Item or Software Module, meaning it must go through safety classification, design controls, and version/update management just like any other software component.
This is not a formality—it is the reviewer’s logic. When you define AI as a Software Item, you are not “explaining neural networks”; you are explaining how you govern a high-risk black-box component so it remains verifiable, traceable, and rollbackable. In practice, if you underestimate the safety class, your documents and testing strategy will be heavily questioned later. For AI involved in diagnosis or treatment decisions, teams often land in Class B or Class C. This classification is not just a regulatory label—it directly determines your evidence cost structure, time-to-market, and how deeply reviewers will probe.
2. Design Control MVP: The Shortest Path to a DHF Skeleton
In practice, “design controls” in IEC 62304 means tying software development and risk control into one line—often forming the skeleton of a DHF (Design History File). Your goal is not a mountain of documents, but a reviewer-walkable path where each step links to evidence. The shortest path can be distilled into:
SRS → SAD → SDS → Implementation → V&V
When implementing this, focus on three reviewer-friendly “design moves.” Reviewers are not counting pages; they are checking whether you translated uncertainty into controllable engineering behaviors.
Move 1: In the SRS, write AI requirements as verifiable system behaviors
In the System Requirements Specification (SRS), do not just say “the AI detects lesions.” Convert uncertainty into system-governed rules. For example: the AI output must include confidence, and only outputs above a pre-defined threshold may be accepted by the system; if confidence is below the threshold, the system must enter a degraded mode or trigger human review. The point is not whether the threshold is 0.8 or 0.75; the point is that you encode risk control as testable system behavior. This turns model uncertainty into deterministic system behavior, so reviewers see risk control as part of design—not a gamble placed on the model.
Move 2: In the SAD, treat AI as a black-box service with strict interface definitions
At the Software Architecture Design (SAD) level, you do not need to expose the model internals, but you must make it a governable service module. At minimum, clearly define boundaries that are inspectable and testable:
- What are the inputs? formats/ranges, including image/signal specs, missing data handling, exceptions
- What preprocessing occurs? which steps are fixed rules vs. steps that affect model output
- What is the output? structured results and required fields (classification/ localization/ confidence/ alerts)
- Interface contract: API signature, versions, compatibility, rollback strategy
- Error codes and timeout handling: what state the system enters on failure/latency
- How the confidence threshold is consumed by the system: UI prompts, logs, audit trails for low-confidence flows
Reviewers need to see you “locked the black-box boundaries.” Even if a model is not fully explainable, strict engineering boundary conditions can keep the system controllable under failure, placing risk control at the system level—not pushing responsibility back to the model.
Move 3: SDS is about standardization, not line-by-line explanation
A minimum viable Software Detailed Design (SDS) should make the AI module an auditable engineering object. You are not writing an academic paper—you are establishing governance so you can answer: How is the model defined, built, and rerun? SDS should include (with consistent version control):
- Model type and constraints (classification/detection/segmentation; pipelines)
- Input/output formats and preprocessing/postprocessing specs (normalization, cropping, denoising, etc.)
- Training data sources and labeling quality control (guidelines, labeler qualifications, inter-rater agreement, exclusion rules)
- Data split strategy (train/validation/test principles; external/cross-site data use)
- Training process governance (hyperparameter versioning, random seeds, training environment, dependencies traceability)
The goal is reproducible governance: any rerun should answer a hard question—is this result reproducible under the same engineering conditions, and auditable?
3. Version & Change Management: Treat Retraining as a Software Update
A common failure mode in AI SaMD is rapid iteration without traceable versions. IEC 62304’s practical solution is straightforward: treat every training/retraining as a software update, with an update plan and rollback strategy. Version labels cannot be only “v1.2”; you must bind the model version to the training dataset version and hyperparameter version, so you can always answer:
- Which model version is currently marketed?
- Which dataset trained it?
- Which hyperparameters were used?
- What changed vs. the previous version?
This is also the shared language across markets: the FDA cares about marketed version clarity, change impacts, and ongoing risk controls; EU MDR cares whether change governance is embedded in QMS and can be audited. For company governance and investors, this section is frequently sampled in post-investment audits and M&A due diligence—because it determines whether your evidence can be trusted.
4. Minimum V&V: Non-AI Uses Unit Tests; AI Uses End-to-End and Performance Evidence
Many teams overcomplicate V&V and end up with processes they cannot maintain. A minimum viable approach aligns with reviewer thinking by keeping the evidence chain clean:
- Non-AI modules: standard unit tests (data pipeline, interfaces, business rules, UI behavior, exception handling)
- AI module: end-to-end tests and performance metrics aligned to claimed clinical use, with metrics tied back to risk-control requirements
- System-level: integration tests and system verification focused on interface protocols, boundary cases, degraded-mode flows, and human review triggers; verify sensitivity/specificity (or key metrics) on clinical or simulated data
The key is not how many tests you ran, but whether you can contain AI uncertainty inside measurable performance metrics and verifiable system behaviors. If the SRS defines confidence thresholds and degraded modes, V&V must prove: performance meets targets at high confidence; low confidence triggers the correct review/degrade behavior; and integrations do not fail at boundaries. Reviewers want to see risk managed through control structures—not through claims.
5. Traceability Matrix MVP: Not a Giant Cross-Table, but an Auditable Evidence Chain
The minimum definition of traceability is one sentence: you must be able to walk a line from risk to evidence. You do not need an unmaintainable “everything-to-everything” matrix; you only need each risk to map to at least one requirement, design item, test case, and evidence artifact.
Strong recommendation: treat one traceability master table as the single source of truth, with minimum columns and a consistent naming convention:
- Risk ID (ISO 14971)
- Requirement ID (SRS)
- Design Item ID (SAD/SDS)
- Test Case ID (V&V)
- Evidence Location (report/file path, accessible and auditable)
A practical example can be expressed as a single chain. For a rare-lesion miss risk: define risk R-001; requirement SR-001 states confidence output and mandatory human review below threshold; design item D-AI-001 implements output + review prompt logic; test case TC-001 validates sensitivity at high confidence on an external dataset and verifies low-confidence triggers review flags; attach evidence AI_Test_Report_v1.0.pdf back into the master table. If this line is complete, you have a minimal but auditable control structure.
6. Make It Maintainable: One Master + Three Subtables, Plus Two AI-Specific Controls
Maintenance cost is often overlooked. A mature MVP approach uses one master table plus three subtables:
- Risk → Requirements (every risk is captured by requirements)
- Requirements → Design (every requirement has a traceable design landing)
- Requirements → Test (every requirement has tests and usable evidence)
Tools can be Excel or ALM systems like Jira, but your SOP must state one thing: every time the AI model is added or modified, you must update the entry points and traceability, otherwise the documentation will break during the next audit or post-market change.
Two AI-specific reviewer questions are almost guaranteed. You can preempt them with minimal extensions that integrate naturally into traceability:
AI Control 1: Black box and explainability
You do not need to promise full explainability, but you must propose reasonable methods and use cases, and include representative-case visualizations/explanations as design and test evidence. You can define explainability outputs in SDS and add an “Explainability Method” column in traceability pointing to the related file/report.
AI Control 2: Data drift and model degradation
Write data monitoring and degradation detection into the SRS, link to monitoring module design and post-market monitoring plans, so reviewers see risk as lifecycle-wide—not only pre-market. Add “Drift Monitoring” or “Post-market Monitoring” columns in traceability with pointers to design and plan documents.
These controls are not for show; they prove you understand AI risk spans the full lifecycle—and you can govern it with engineering discipline.
Conclusion (VC Perspective)
From a MedTech VC diligence lens, the ability to implement IEC 62304 in a practical way is a scarce competitive advantage—not merely a compliance cost. The teams that ship fast and survive long-term are often not the ones who can talk best about model architectures, but the ones who can execute engineering governance.
Can you define AI clearly as a Software Item?
Can you translate uncertainty into verifiable system behavior?
Can you govern retraining as a traceable software update with rollback?
Can you lock the risk-to-evidence chain with minimal traceability?
These determine whether your development pace is predictable, whether clinical and regulatory evidence can stack smoothly, and whether in M&A due diligence you can precisely explain which model version produced those “great numbers.” For investors, the MVP architecture does not buy documents—it buys certainty: compressing regulatory and delivery risk into manageable engineering risk, making the time and cash burn curve highly controllable. That is the underlying logic for an AI SaMD valuation that can truly hold.
If you’d like to learn more, please feel free to email Joseph.Shen@behealthventures.com.
