Welcome to chapter twenty. Software is one tool in your kit, alongside the workshop and the electronics bench, and in a hospital it is among the most consequential. Hospitals run on software: the health record, the image archive, the laboratory system, the pump's drug library, the monitor's alarm logic. And software arrives uninvited. Put a microcontroller in a task trainer, a sensor on an assistive aid, or a radio link in a wearable, and you have written firmware, you may be handling data about a person, and you have a device that can be attacked. This chapter is about all of that.
Health software is not ordinary software with a medical theme. Four things change. First, software can be a medical device, and whether it is depends not on the code but on what you say it is for. Second, the data are special: health data are among the most sensitive personal data there are, and protected by law in every jurisdiction you are likely to work in. Third, systems must interoperate through standards with long histories and sharp edges. And fourth, security failures harm patients directly. A word of caution: this chapter is orientation, summarised as things stood in twenty twenty-six. What applies to a real project is settled with the host's governance, clinical safety and regulatory staff.
Let's start with the question that changes everything else: is this software a medical device?
A medical device is defined by its intended purpose: what the manufacturer claims it is for. Take one algorithm that analyses heart rate from a wrist sensor. Marketed as "track your workouts", it is a wellness product and generally not a device. Marketed as notifying you of an irregular rhythm suggestive of atrial fibrillation, it is a medical device, and smartwatch features like that went through regulatory clearance for exactly this reason. Used in a hospital to decide who receives anticoagulation, it is higher risk still. Regulators look at promotion, reasonable use and the claims on the screen, not just the label. A footer saying "educational purposes only" does not neutralise a screen that announces a diagnosis.
Two terms from the International Medical Device Regulators Forum. Software as a Medical Device is software that is itself the device, running on ordinary hardware: an app that grades diabetic retinopathy from photographs, an algorithm that flags bleeding on a head CT, an insulin dose calculator. Software in a Medical Device drives or is part of hardware: the firmware in an infusion pump, the reconstruction software in a CT scanner, the control loop of a ventilator. Both are regulated, along different paths. Software that is neither, like scheduling or billing, is usually not a device, but it can still be safety-critical. In England, clinical risk management standards apply to health IT whether or not it is a device, and require a named Clinical Safety Officer.
The IMDRF framework classifies software with two questions. How serious is the healthcare situation? And how much does the clinician or patient rely on the software's output? Inform means the output is one input among many to a clinician's decision. Drive means it guides the next step. Treat or diagnose means the output itself is the decision. Category four, critical situations where the software diagnoses or treats, is the highest risk. Regulators do not all use this table directly, but most risk-based software classification echoes its logic, and it is the fastest way to reason about where a project sits.
The main regimes in outline. In the United States the FDA regulates device software functions; the twenty-first Century Cures Act excluded some functions, including certain clinical decision support that lets a clinician independently review the basis of the recommendation. In the European Union, Rule eleven of the Medical Device Regulation puts software that informs diagnostic or therapeutic decisions at Class two A or higher, so very little clinical software is Class one. The United Kingdom has its own regulator and NHS standards on top. The grey zone you are most likely to wander into is decision support. A rule that says "consider sepsis screening" to a clinician who can see why it fired is treated very differently from one that says "start antibiotics". Settle that before you write the rule.
Once software is a device, a defined lifecycle applies: IEC sixty-two three oh four.
Chapter nineteen introduced risk management and usability engineering. IEC sixty-two three oh four is the third member of that family. It does not tell you how to write code; it tells you what you must plan, do and record. Planning, software requirements, architecture, detailed design, unit verification, integration testing, system testing and release, then maintenance, where every change goes back through the process. Running throughout are software risk management, configuration management and problem resolution, all inside a quality management system. And iterative development is allowed: agile is fine, provided "done" includes the records.
Each software system is given a safety class by the harm a failure could contribute to, assessed after controls outside the software. Class A: no injury is possible. Class B: non-serious injury is possible. Class C: death or serious injury is possible, with the most demanding process. Two architectural consequences follow. First, an independent hardware control, like a mechanical pressure-relief valve or a hardware interlock, can lower the software's class, because the software can no longer cause the harm on its own. Second, segregation lets you confine class C to a small, heavily verified component, provided the segregation is real and documented, not just a folder name.
SOUP stands for software of unknown provenance: anything you did not develop under your process, from the operating system to the package with eleven transitive dependencies. For each one you record its version, the requirements you rely on it for, what it needs to run, and an evaluation of its published bugs for effects on safety. That is the regulatory form of something you already know: you are responsible for what you ship, including what you imported. Configuration management means you can say exactly what was in a release and rebuild it. Problem resolution means every reported problem is recorded, assessed, resolved and verified. Your issue tracker and git history are most of this, if you use them with discipline.
Now the lessons that were written in harm, starting with the most famous one.
Between nineteen eighty-five and nineteen eighty-seven, the Therac twenty-five radiotherapy machine gave massive overdoses to at least six patients, and several died. Almost every failure was a software engineering failure rather than a coding slip. The software could set the machine up inconsistently after a fast edit of the prescription. The independent hardware interlock that earlier models had was removed, and software was trusted instead. The operator resumed treatment after a cryptic error message that appeared so often it was routinely overridden. And the manufacturer insisted an overdose was impossible, so early reports were not investigated, and the accidents recurred.
Each lesson has a modern equivalent. Removing a hardware interlock moved the whole safety burden onto code that was never designed or verified for it. The race condition appeared only when an experienced operator edited the prescription quickly, so testing at normal pace never found it. Frequent, uninformative errors trained operators to override them, which is alarm fatigue. Reused code was assumed safe because it had worked in an earlier machine, where the hardware interlocks had been masking its faults. And overconfidence delayed the response. Leveson and Turner's investigation is required reading for anyone whose software can hurt people.
Infusion pumps generated tens of thousands of adverse event reports, with software defects and interface problems among the major contributors. A missed decimal point turns one into ten. Units get confused, implausible values are silently clamped, and users think the pump is in one mode when it is in another. The response is dose error reduction software: a drug library with hard and soft limits, which only works if it is kept current and not bypassed. Unit errors recur because numbers travel through code without their units; glucose units differ by a factor of eighteen. And alarms: studies repeatedly find most monitor alarms are false or clinically irrelevant. An alarm is a diagnostic test with a prevalence problem, and people respond accordingly.
Health data are any data that reveal something about a person's health. That is wider than it looks: an appointment at a fertility clinic, a dietary requirement, a step count linked to a name, a wound photograph, a DICOM image header. Identifiability is a spectrum. Identified data carry names and record numbers. Pseudonymised data replace identifiers with a key held elsewhere, and are still personal data in law. De-identified data have had direct identifiers removed and quasi-identifiers reduced. Anonymous is a high bar. Deleting names is the easy part: a widely cited study estimated that about eighty-seven percent of Americans are unique on just five-digit ZIP code, sex and full date of birth.
Medical data add their own traps. Free-text notes mention names, relatives, employers and places. Dates of admission, procedures and birth are strongly identifying; shifting them consistently per patient preserves intervals while hiding the calendar. Images carry burned-in text, faces that can be reconstructed from head scans, and DICOM headers holding names, dates, institutions and private vendor tags. Rare conditions in small places identify people on their own. And a dataset that is safe alone can identify people once it is joined with another, including a dataset that has not been published yet.
In outline. Under GDPR, health, genetic and biometric data are special category data: processing needs an ordinary lawful basis and a specific Article nine condition, and consent is only one possible basis, often not the one used for direct care. High-risk processing needs a data protection impact assessment before it starts. In the UK, the Caldicott principles add justified purpose, minimum necessary and need-to-know. In the US, HIPAA applies to covered entities and their business associates, with de-identification by Safe Harbor, removing eighteen listed identifiers, or by expert determination. And one absolute rule: never put real patient data into a tool the organisation has not approved. Once it has left the boundary, it is a breach, and it cannot be recalled.
A hospital may run hundreds of clinical systems, exchanging data through a few standards. HL7 version two is decades old, pipe-delimited, and still carries most hospital traffic, but every site customises it. FHIR is the modern standard: resources like Patient and Observation as JSON over a REST interface, with SMART on FHIR adding app launch and authorisation. DICOM is both the file format and the network protocol for images. And the meaning lives in the codes: SNOMED CT for clinical findings, LOINC for what was measured, ICD for diagnoses, and UCUM for units. In a FHIR Observation, the display text "heart rate" is for humans; the LOINC code is what a receiving system acts on. Status, time and subject are part of the measurement too.
Now security, where the stakes are clinical, not just commercial.
Hospitals combine everything an attacker wants: valuable data, a critical service that must be restored quickly, large flat networks, thousands of busy users, and long-lived devices running old operating systems that cannot easily be patched. In twenty seventeen, WannaCry disrupted a third of hospital trusts in England, cancelling appointments and diverting ambulances. In twenty twenty-four, ransomware at Change Healthcare disrupted pharmacy and payment processing across the United States for weeks, and an attack on Synnovis halted pathology for several London hospitals, forcing cancelled operations and appeals for blood donors. The lesson is that availability is a safety property.
Since March twenty twenty-three, US premarket submissions for internet-connectable devices must include a plan to monitor and address vulnerabilities, coordinated vulnerability disclosure, and a software bill of materials. Europe builds IT security into its safety requirements, and IEC eighty-one thousand and one, part five one, defines a secure development lifecycle. In practice: threat model the clinical workflow, not just the code. Controls that obstruct care at three in the morning get bypassed, and the bypass becomes the real security posture. Plan for patching from the start. And for the hardware you build: close the debug port, sign firmware updates, avoid hard-coded secrets, and define what the device does when its link fails.
Building a model that performs well on a test set is now easy. Building one that works in a different hospital next year is not. Dataset shift: patients, scanners, protocols and prevalence differ between sites, and a widely deployed sepsis model performed substantially worse on external validation than its developer's figures. Shortcut learning: a pneumonia model learned to recognise which hospital and which X-ray machine produced an image, rather than the lungs. Good average performance can hide poor performance in a subgroup. Labels are often one clinician's opinion or a billing code. And leakage and prevalence mismatch inflate results. Regulators distinguish locked from adaptive algorithms, and the FDA now allows predetermined change control plans.
Three habits from the chapter's code examples. Carry the unit in the type, convert once at the boundary, and reject implausible or ambiguous input loudly rather than clamping or guessing. But rejection alone is not enough. A pump that silently refuses ten is as dangerous as one that accepts it, if the user does not notice. Say what is wrong and what to do, distinguish impossible values from unusual ones that need a deliberate confirmation, never auto-correct a clinical value, and read the value back with its unit before committing it. For audit trails, chaining each entry to the hash of the previous one makes tampering detectable, so you can show the record has not been changed.
A dataset is k-anonymous if every combination of quasi-identifiers is shared by at least k records. Take a thousand fictional patients described by sex, year of birth across eighty years, and one of fifty postcode districts. That is eight thousand possible combinations, eight times more cells than people, so most people sit alone in their cell: k equals one, and the dataset identifies nearly everyone to anyone who knows those three facts. Generalise to ten-year age bands and five regions, and you get eighty cells, averaging about twelve people each. But k is a floor, not a guarantee: if everyone in a cell shares a diagnosis, the diagnosis is disclosed anyway.
Threat modelling with STRIDE. Draw the data flows, mark the trust boundaries, then ask six questions of each element. Spoofing: can someone submit answers as another learner? Tampering: can a result be changed after submission, and who could tell? Repudiation: can the educator show who changed a record? Information disclosure: can one learner see another's results, or is anything identifying in a URL or log? Denial of service: what happens if the server is unreachable? Elevation of privilege: can a learner reach the educator's dashboard? Each answer becomes a control with a test, or a documented, accepted risk. And the fault tree logic from Therac twenty-five generalises: add an independent barrier so a software fault alone cannot cause harm.
You can learn almost all of this without a single real record. Synthea generates synthetic patients exportable as FHIR. HAPI FHIR gives you a local FHIR server in minutes, and Inferno shows what conformance actually tests. Pydicom and defacing tools let you inspect and de-identify image files, including private tags. ARX and sdcMicro measure k-anonymity and re-identification risk. Threat Dragon draws data flows and enumerates STRIDE threats, and Syft, CycloneDX and OSV-Scanner build bills of materials and match them against known vulnerabilities. Public physiological and imaging data exist too; some, like MIMIC, require credentialing, which you must honour exactly.
The worked example. The educator asks for a small app: learners scan a QR code on the elbow model, answer three questions, and results go to the educator. Version one is not a device, the personal data is minimal, and security is ordinary. Then: could it also record whether the learner has joint hypermobility? That is health data about identifiable students for a research-like purpose. Stop, write the question down, send it to the educator and the governance team, and do not add the field. Then: could patients use it at home to measure their elbow angle? Now it monitors a condition and informs management, very probably a device. The right response is neither refusal nor quiet building: it is a different product, needing a sponsor and a regulatory decision. Scope creep is regulatory creep.
For the exercises. Classify five apps on the IMDRF table, from a step counter to a sepsis alert, and name the question that would decide each one. Generate synthetic patients, load them into a local FHIR server, and write a script that retrieves blood pressure observations with units and timestamps, then make it reject a missing or wrong unit. De-identify a public DICOM file and document what remained, including any burned-in text. Compute k for three generalisations of a synthetic table. And draw a data-flow diagram and STRIDE table for the classroom mock monitor, then generate a software bill of materials for one of your own projects.
The limits. You can reason about where a product probably sits, but the determination belongs to the manufacturer's regulatory function and ultimately the regulator. Knowing the shape of the standards is not compliance with them. Nothing here is legal advice; data protection law differs by jurisdiction and changes often. De-identification is specialist work, and a declaration of anonymity can be overturned by new linkable data. Never probe or test a hospital system or a medical device you do not own without explicit written permission; a crashed device is a patient safety incident. And whether an algorithm helps patients is a clinical question, answered in the setting of use. Next, making a model that teaches.