Chapter 20
Health data, medical software, and security When code becomes a device, why the data are special, and why availability is safety
Human Biology for the Medical Maker
Welcome to chapter twenty. Software is one tool in your kit, alongside the workshop and the electronics bench, and in a hospital it is among the most consequential. Hospitals run on software: the health record, the image archive, the laboratory system, the pump's drug library, the monitor's alarm logic. And software arrives uninvited. Put a microcontroller in a task trainer, a sensor on an assistive aid, or a radio link in a wearable, and you have written firmware, you may be handling data about a person, and you have a device that can be attacked. This chapter is about all of that. Four things change in health software What changes Why it matters Software can be a medical device Decided by what you say it is for ; then IEC 62304 applies The data are special Protected by law everywhere; de-identification is hard Systems must interoperate HL7, FHIR, DICOM — and codes and units carry the meaning Security failures harm patients Ransomware has closed emergency departments
Orientation, not legal or regulatory advice. What applies is decided with the host's specialists.
Health software is not ordinary software with a medical theme. Four things change. First, software can be a medical device, and whether it is depends not on the code but on what you say it is for. Second, the data are special: health data are among the most sensitive personal data there are, and protected by law in every jurisdiction you are likely to work in. Third, systems must interoperate through standards with long histories and sharp edges. And fourth, security failures harm patients directly. A word of caution: this chapter is orientation, summarised as things stood in twenty twenty-six. What applies to a real project is settled with the host's governance, clinical safety and regulatory staff. When is software a medical device? Intended use decides
Let's start with the question that changes everything else: is this software a medical device? One algorithm, three products Marketed as Status "Track your workouts" General wellness — generally not a device "Notifies you of an irregular rhythm suggestive of atrial fibrillation" A medical device Used in hospital to decide who gets anticoagulation A higher-risk device still
"For educational purposes only" in a footer does not neutralise a screen that says "Diagnosis: atrial fibrillation".
A medical device is defined by its intended purpose: what the manufacturer claims it is for. Take one algorithm that analyses heart rate from a wrist sensor. Marketed as "track your workouts", it is a wellness product and generally not a device. Marketed as notifying you of an irregular rhythm suggestive of atrial fibrillation, it is a medical device, and smartwatch features like that went through regulatory clearance for exactly this reason. Used in a hospital to decide who receives anticoagulation, it is higher risk still. Regulators look at promotion, reasonable use and the claims on the screen, not just the label. A footer saying "educational purposes only" does not neutralise a screen that announces a diagnosis. SaMD and SiMD Software as a Medical Device Is the device, on general-purpose hardware Retinopathy grading from photographs Haemorrhage flagging on CT; insulin dose calculators Software in a Medical Device Drives or is part of a hardware device Infusion pump firmware CT reconstruction; ventilator control loops "Not a device" does not mean "not a safety question" — England's DCB0129 and DCB0160 apply to health IT anyway.
Two terms from the International Medical Device Regulators Forum. Software as a Medical Device is software that is itself the device, running on ordinary hardware: an app that grades diabetic retinopathy from photographs, an algorithm that flags bleeding on a head CT, an insulin dose calculator. Software in a Medical Device drives or is part of hardware: the firmware in an infusion pump, the reconstruction software in a CT scanner, the control loop of a ventilator. Both are regulated, along different paths. Software that is neither, like scheduling or billing, is usually not a device, but it can still be safety-critical. In England, clinical risk management standards apply to health IT whether or not it is a device, and require a named Clinical Safety Officer. The IMDRF risk framework Healthcare situation Treat or diagnose Drive management Inform management Critical IV III II Serious III II I Non-serious II I I
Two questions: how serious is the situation, and how much does anyone rely on the output?
The IMDRF framework classifies software with two questions. How serious is the healthcare situation? And how much does the clinician or patient rely on the software's output? Inform means the output is one input among many to a clinician's decision. Drive means it guides the next step. Treat or diagnose means the output itself is the decision. Category four, critical situations where the software diagnoses or treats, is the highest risk. Regulators do not all use this table directly, but most risk-based software classification echoes its logic, and it is the fastest way to reason about where a project sits. The shape of the main regimes Jurisdiction In outline United States FDA "device software functions"; Cures Act exclusions incl. some clinical decision support; quality regulation incorporates ISO 13485 from February 2026 European Union MDR Rule 11: decision-informing software at least Class IIa; little clinical software is Class I. EU AI Act phasing in to 2027 United Kingdom MHRA regulation under reform; NHS adds DCB0129, DCB0160, and DTAC
Clinical decision support is the grey zone you are most likely to wander into. Settle it before writing the rule.
The main regimes in outline. In the United States the FDA regulates device software functions; the twenty-first Century Cures Act excluded some functions, including certain clinical decision support that lets a clinician independently review the basis of the recommendation. In the European Union, Rule eleven of the Medical Device Regulation puts software that informs diagnostic or therapeutic decisions at Class two A or higher, so very little clinical software is Class one. The United Kingdom has its own regulator and NHS standards on top. The grey zone you are most likely to wander into is decision support. A rule that says "consider sepsis screening" to a clinician who can see why it fired is treated very differently from one that says "start antibiotics". Settle that before you write the rule. The software lifecycle — IEC 62304 What you must plan, do, and record
Once software is a device, a defined lifecycle applies: IEC sixty-two three oh four. The IEC 62304 lifecycle Development activities, maintenance, and the supporting processes that run throughout
Chapter nineteen introduced risk management and usability engineering. IEC sixty-two three oh four is the third member of that family. It does not tell you how to write code; it tells you what you must plan, do and record. Planning, software requirements, architecture, detailed design, unit verification, integration testing, system testing and release, then maintenance, where every change goes back through the process. Running throughout are software risk management, configuration management and problem resolution, all inside a quality management system. And iterative development is allowed: agile is fine, provided "done" includes the records. Software safety classes Class Worst outcome if the software fails Process burden A No injury possible Planning, requirements, release, configuration, problem resolution B Non-serious injury possible Adds architecture, unit, integration, and system testing C Death or serious injury possible Adds detailed design and the most rigorous unit verification
An independent hardware control can lower the class. Segregation can confine class C to a small component.
Each software system is given a safety class by the harm a failure could contribute to, assessed after controls outside the software. Class A: no injury is possible. Class B: non-serious injury is possible. Class C: death or serious injury is possible, with the most demanding process. Two architectural consequences follow. First, an independent hardware control, like a mechanical pressure-relief valve or a hardware interlock, can lower the software's class, because the software can no longer cause the harm on its own. Second, segregation lets you confine class C to a small, heavily verified component, provided the segregation is real and documented, not just a folder name. SOUP, configuration, and problem resolution SOUP : software of unknown provenance — every dependency you did not buildRecord version, what you rely on it for, and its published bugs Be able to reproduce exactly what was in a release Every reported problem recorded, assessed for safety, resolved, verified You already know this — you are responsible for what you ship, including what you imported.
SOUP stands for software of unknown provenance: anything you did not develop under your process, from the operating system to the package with eleven transitive dependencies. For each one you record its version, the requirements you rely on it for, what it needs to run, and an evaluation of its published bugs for effects on safety. That is the regulatory form of something you already know: you are responsible for what you ship, including what you imported. Configuration management means you can say exactly what was in a release and rebuild it. Problem resolution means every reported problem is recorded, assessed, resolved and verified. Your issue tracker and git history are most of this, if you use them with discipline. What goes wrong Lessons written in harm
Now the lessons that were written in harm, starting with the most famous one. Therac-25 as a fault tree A software fault AND no hardware interlock AND a routinely overridden error — with an organisation that let it recur
Between nineteen eighty-five and nineteen eighty-seven, the Therac twenty-five radiotherapy machine gave massive overdoses to at least six patients, and several died. Almost every failure was a software engineering failure rather than a coding slip. The software could set the machine up inconsistently after a fast edit of the prescription. The independent hardware interlock that earlier models had was removed, and software was trusted instead. The operator resumed treatment after a cryptic error message that appeared so often it was routinely overridden. And the manufacturer insisted an overdose was impossible, so early reports were not investigated, and the accidents recurred. The lessons of Therac-25 Do not replace an independent interlock with software Concurrency faults can depend on operator speed Frequent cryptic errors train users to override them Reused code is not proven safe by another system's history Investigate incident reports; do not reassure them away Required reading: Leveson and Turner's investigation.
Each lesson has a modern equivalent. Removing a hardware interlock moved the whole safety burden onto code that was never designed or verified for it. The race condition appeared only when an experienced operator edited the prescription quickly, so testing at normal pace never found it. Frequent, uninformative errors trained operators to override them, which is alarm fatigue. Reused code was assumed safe because it had worked in an earlier machine, where the hardware interlocks had been masking its faults. And overconfidence delayed the response. Leveson and Turner's investigation is required reading for anyone whose software can hurt people. Pumps, units, and alarms Infusion pump failures 1.0 entered as 10: a tenfold overdose mg versus micrograms; per hour versus per minute Silent clamping; mode confusion Response: drug libraries with hard and soft limits Units and alarms Glucose units differ by a factor of 18 Mars Climate Orbiter lost to pound-force versus newtons Most monitor alarms are false or irrelevant An alarm is a test with a prevalence problem Infusion pumps generated tens of thousands of adverse event reports, with software defects and interface problems among the major contributors. A missed decimal point turns one into ten. Units get confused, implausible values are silently clamped, and users think the pump is in one mode when it is in another. The response is dose error reduction software: a drug library with hard and soft limits, which only works if it is kept current and not bypassed. Unit errors recur because numbers travel through code without their units; glucose units differ by a factor of eighteen. And alarms: studies repeatedly find most monitor alarms are false or clinically irrelevant. An alarm is a diagnostic test with a prevalence problem, and people respond accordingly. The identifiability spectrum Identified, pseudonymised, de-identified, anonymous — pseudonymised data are still personal data
Quasi-identifiers combine: ZIP code, sex, and full birth date identify about 87 percent of Americans.
Health data are any data that reveal something about a person's health. That is wider than it looks: an appointment at a fertility clinic, a dietary requirement, a step count linked to a name, a wound photograph, a DICOM image header. Identifiability is a spectrum. Identified data carry names and record numbers. Pseudonymised data replace identifiers with a key held elsewhere, and are still personal data in law. De-identified data have had direct identifiers removed and quasi-identifiers reduced. Anonymous is a high bar. Deleting names is the easy part: a widely cited study estimated that about eighty-seven percent of Americans are unique on just five-digit ZIP code, sex and full date of birth. Medical data's own traps Free text: notes mention names, relatives, employers, places Dates: strongly identifying; shift consistently per patient Images: burned-in text, reconstructable faces, DICOM headers and private tags Rare conditions and small numbers Linkage with another dataset — including one that does not exist yet Medical data add their own traps. Free-text notes mention names, relatives, employers and places. Dates of admission, procedures and birth are strongly identifying; shifting them consistently per patient preserves intervals while hiding the calendar. Images carry burned-in text, faces that can be reconstructed from head scans, and DICOM headers holding names, dates, institutions and private vendor tags. Rare conditions in small places identify people on their own. And a dataset that is safe alone can identify people once it is joined with another, including a dataset that has not been published yet. The legal frameworks in outline Framework What a builder needs to know GDPR and UK GDPR Health data are special category: a lawful basis and an Article 9 condition; DPIA for high-risk processing; data protection by design Caldicott principles (UK) Justify the purpose; minimum necessary; need-to-know; the duty to share matters too HIPAA (US) Covered entities and business associates; minimum necessary; de-identify by Safe Harbor (18 identifiers) or Expert Determination
Never put real patient data into a tool the organisation has not approved — including a chat assistant or a bug report.
In outline. Under GDPR, health, genetic and biometric data are special category data: processing needs an ordinary lawful basis and a specific Article nine condition, and consent is only one possible basis, often not the one used for direct care. High-risk processing needs a data protection impact assessment before it starts. In the UK, the Caldicott principles add justified purpose, minimum necessary and need-to-know. In the US, HIPAA applies to covered entities and their business associates, with de-identification by Safe Harbor, removing eighteen listed identifiers, or by expert determination. And one absolute rule: never put real patient data into a tool the organisation has not approved. Once it has left the boundary, it is a breach, and it cannot be recalled. The standards Standard What to know HL7 version 2 Pipe-delimited messages; still carries most hospital traffic; every site customises it FHIR Resources as JSON over REST; R4 widely deployed; SMART on FHIR for app launch DICOM File format and network protocol for images; DICOMweb over HTTP Terminologies SNOMED CT findings; LOINC what was measured; ICD diagnoses; UCUM units
A number without a code and a unit is not clinical data.
A hospital may run hundreds of clinical systems, exchanging data through a few standards. HL7 version two is decades old, pipe-delimited, and still carries most hospital traffic, but every site customises it. FHIR is the modern standard: resources like Patient and Observation as JSON over a REST interface, with SMART on FHIR adding app launch and authorisation. DICOM is both the file format and the network protocol for images. And the meaning lives in the codes: SNOMED CT for clinical findings, LOINC for what was measured, ICD for diagnoses, and UCUM for units. In a FHIR Observation, the display text "heart rate" is for humans; the LOINC code is what a receiving system acts on. Status, time and subject are part of the measurement too. Security Availability is a safety property
Now security, where the stakes are clinical, not just commercial. Why hospitals are targets Event What happened WannaCry (2017) Unpatched Windows worldwide; a third of NHS trusts in England disrupted, ambulances diverted Change Healthcare (2024) US claims and pharmacy processing disrupted for weeks; largest reported US health data breach Synnovis (2024) London pathology halted; operations cancelled; appeals for O-type blood donors
A system that is confidential and intact but down is a system whose patients are not being treated.
Hospitals combine everything an attacker wants: valuable data, a critical service that must be restored quickly, large flat networks, thousands of busy users, and long-lived devices running old operating systems that cannot easily be patched. In twenty seventeen, WannaCry disrupted a third of hospital trusts in England, cancelling appointments and diverting ambulances. In twenty twenty-four, ransomware at Change Healthcare disrupted pharmacy and payment processing across the United States for weeks, and an attack on Synnovis halted pathology for several London hospitals, forcing cancelled operations and appeals for blood donors. The lesson is that availability is a safety property. What regulators expect, and secure development applied Regulators US cyber devices since March 2023: vulnerability plan, coordinated disclosure, SBOM EU MDR: IT security in safety requirements IEC 81001-5-1 secure lifecycle; MDS2 form Applied Threat model the clinical workflow: gloves, 3 a.m., emergencies Plan for patching: signed updates, rollback Embedded: close debug ports, no hard-coded secrets, a defined safe state Since March twenty twenty-three, US premarket submissions for internet-connectable devices must include a plan to monitor and address vulnerabilities, coordinated vulnerability disclosure, and a software bill of materials. Europe builds IT security into its safety requirements, and IEC eighty-one thousand and one, part five one, defines a secure development lifecycle. In practice: threat model the clinical workflow, not just the code. Controls that obstruct care at three in the morning get bypassed, and the bypass becomes the real security posture. Plan for patching from the start. And for the hardware you build: close the debug port, sign firmware updates, avoid hard-coded secrets, and define what the device does when its link fails. AI and machine learning Dataset shift: a widely used sepsis model performed far worse at a new site Shortcut learning: a pneumonia model learned which hospital took the X-ray Subgroup performance hidden by the average Labels are often one opinion or a billing code Leakage and prevalence mismatch inflate results Locked versus adaptive algorithms; predetermined change control plans (FDA, December 2024).
Building a model that performs well on a test set is now easy. Building one that works in a different hospital next year is not. Dataset shift: patients, scanners, protocols and prevalence differ between sites, and a widely deployed sepsis model performed substantially worse on external validation than its developer's figures. Shortcut learning: a pneumonia model learned to recognise which hospital and which X-ray machine produced an image, rather than the lungs. Good average performance can hide poor performance in a subgroup. Labels are often one clinician's opinion or a billing code. And leakage and prevalence mismatch inflate results. Regulators distinguish locked from adaptive algorithms, and the FDA now allows predetermined change control plans. Units, validation, and audit trails Unit is part of the type; convert once at the boundary; reject ambiguous input Validation that respects the user: say what is wrong and what to do Impossible means hard stop; unusual means deliberate, logged confirmation Never auto-correct a clinical value; read back with the unit Hash-chained audit logs make tampering detectable Three habits from the chapter's code examples. Carry the unit in the type, convert once at the boundary, and reject implausible or ambiguous input loudly rather than clamping or guessing. But rejection alone is not enough. A pump that silently refuses ten is as dangerous as one that accepts it, if the user does not notice. Say what is wrong and what to do, distinguish impossible values from unusual ones that need a deliberate confirmation, never auto-correct a clinical value, and read the value back with its unit before committing it. For audit trails, chaining each entry to the hash of the previous one makes tampering detectable, so you can show the record has not been changed. k-anonymity arithmetic k = 1
1,000 people × sex, birth year, postcode district = 8,000 cells — most people are alone in theirs
Generalise to 10-year bands and regions: 80 cells, about 12 per cell. k is a floor, not a guarantee.
A dataset is k-anonymous if every combination of quasi-identifiers is shared by at least k records. Take a thousand fictional patients described by sex, year of birth across eighty years, and one of fifty postcode districts. That is eight thousand possible combinations, eight times more cells than people, so most people sit alone in their cell: k equals one, and the dataset identifies nearly everyone to anyone who knows those three facts. Generalise to ten-year age bands and five regions, and you get eighty cells, averaging about twelve people each. But k is a floor, not a guarantee: if everyone in a cell shares a diagnosis, the diagnosis is disclosed anyway. Threat modelling with STRIDE S poofing — authenticationT ampering — integrityR epudiation — who changed whatI nformation disclosure — confidentialityD enial of service — availabilityE levation of privilege — authorisationThreat modelling with STRIDE. Draw the data flows, mark the trust boundaries, then ask six questions of each element. Spoofing: can someone submit answers as another learner? Tampering: can a result be changed after submission, and who could tell? Repudiation: can the educator show who changed a record? Information disclosure: can one learner see another's results, or is anything identifying in a URL or log? Denial of service: what happens if the server is unreachable? Elevation of privilege: can a learner reach the educator's dashboard? Each answer becomes a control with a test, or a documented, accepted risk. And the fault tree logic from Therac twenty-five generalises: add an independent barrier so a software fault alone cannot cause harm. Tools you can use today Tool Use Synthea Synthetic patient records as FHIR — never develop against real data HAPI FHIR, Inferno Local FHIR server; conformance testing pydicom, defacing tools Inspect and de-identify DICOM, including private tags ARX, sdcMicro k-anonymity and re-identification risk metrics Threat Dragon; Syft, CycloneDX, OSV-Scanner Threat models; SBOMs and vulnerability matching PhysioNet, TCIA Public data — honour any credentialing exactly
You can learn almost all of this without a single real record. Synthea generates synthetic patients exportable as FHIR. HAPI FHIR gives you a local FHIR server in minutes, and Inferno shows what conformance actually tests. Pydicom and defacing tools let you inspect and de-identify image files, including private tags. ARX and sdcMicro measure k-anonymity and re-identification risk. Threat Dragon draws data flows and enumerates STRIDE threats, and Syft, CycloneDX and OSV-Scanner build bills of materials and match them against known vulnerabilities. Public physiological and imaging data exist too; some, like MIMIC, require credentialing, which you must honour exactly. Worked example: the companion app that grew Version The request What changes 1 Learners scan a QR code, answer three questions, results to the educator Not a device; minimal personal data; ordinary security 2 Also record whether the learner has joint hypermobility Special category health data; a study — stop, ask, do not add the field 3 Patients measure elbow angle at home with the phone camera Very probably a device; IEC 62304; accuracy validation; a sponsor and regulatory decision
In software, scope creep is regulatory creep. Keep the intended-use statement in the README.
The worked example. The educator asks for a small app: learners scan a QR code on the elbow model, answer three questions, and results go to the educator. Version one is not a device, the personal data is minimal, and security is ordinary. Then: could it also record whether the learner has joint hypermobility? That is health data about identifiable students for a research-like purpose. Stop, write the question down, send it to the educator and the governance team, and do not add the field. Then: could patients use it at home to measure their elbow angle? Now it monitors a condition and informs management, very probably a device. The right response is neither refusal nor quiet building: it is a different product, needing a sponsor and a regulatory decision. Scope creep is regulatory creep. Workshop exercise and exit criteria Classify five apps on the IMDRF table and name the deciding question Build a synthetic FHIR pipeline that handles codes and units explicitly De-identify a public DICOM file and document what remained Compute k for three generalisations of a Synthea table Threat-model the mock monitor; produce an SBOM for a project of yours For the exercises. Classify five apps on the IMDRF table, from a step counter to a sepsis alert, and name the question that would decide each one. Generate synthetic patients, load them into a local FHIR server, and write a script that retrieves blood pressure observations with units and timestamps, then make it reject a missing or wrong unit. De-identify a public DICOM file and document what remained, including any burned-in text. Compute k for three generalisations of a synthetic table. And draw a data-flow diagram and STRIDE table for the classroom mock monitor, then generate a software bill of materials for one of your own projects. What this chapter does not establish You cannot determine regulatory status You are not compliant with IEC 62304 or any other standard This is not legal advice De-identification is specialist work Security testing of clinical systems needs written authorisation A model's metrics are not its clinical value Next — Making a model that teaches.
The limits. You can reason about where a product probably sits, but the determination belongs to the manufacturer's regulatory function and ultimately the regulator. Knowing the shape of the standards is not compliance with them. Nothing here is legal advice; data protection law differs by jurisdiction and changes often. De-identification is specialist work, and a declaration of anonymity can be overturned by new linkable data. Never probe or test a hospital system or a medical device you do not own without explicit written permission; a crashed device is a patient safety incident. And whether an algorithm helps patients is a clinical question, answered in the setting of use. Next, making a model that teaches. 
