Ciara Ferrier

Heidi AI Scribe is registered as a Class I medical device under MHRA guidance for summarisation functionality. The Heidi website states that as the company explore “new and innovative features” their “regulatory footprint will increase, which is why we are progressing what Class II certification would look like for Heidi.”

With its current functionality, some clinicians feel almost affronted that Heidi would be classed as a medical device. This is because the superiority of clinical knowledge is so ingrained amongst them. Of course they are going to review content. Of course any output will reflect their own judgement! Putting Heidi through this additional layer of bureaucracy feels like an affront to their integrity.

Several years ago I worked in medical software development building maternity information systems. A patient record feels like it would just be a series of boxes that the clinician will fill in, ergo it is not a medical device. However, as soon as you put a calculation in, such as deriving age from date of birth, you allow the system to make a judgement and the threshold for needing to comply to medical device standards is really that low.

With an administration/technical background, I have no issue with that rule. You probably are doing the right thing, if you feel affronted by it, but actually this protects your integrity by maintaining that standard across all your colleagues and holding developers to account.

Heidi maybe doesn’t influence your thinking but simply by being able to summarise, it can and that is the key point. The rest of us can relate to this in terms of AI internet search summaries. Do you always look at the links in the summary, check what they say and question the veracity of the source?

So, I view the MHRA guidance on AI scribes as some level of guidance on the protection of critical thinking. Many of us, myself included, will say, “Oh, it’s got ISO 27001 so your data is safe.” But it isn’t actually just about keeping that data safe; it is about the safety of how we THINK about that data.

I’m not aware of a similar device standard applied to products in other sectors such as academic research, law, accountancy, engineering, journalism and so on – please let me know if you are aware of one.  I put my hands up to needing to read more on the EU AI Act, however from initial reading I would expect more products will ultimately fall under those deemed high risk in Annex III.

Medicine already has this more stringent check in place. It is simply asking other sectors to come up to a similar standard and protect their professional judgement and critical thinking. Medics already have the self-worth to know that their opinion should take precedence. The rest of us should do the same.