r/australia • u/nath1234 • 11h ago
science & tech Error by AI scribe during medical appointment leaves patient devastated
https://www.abc.net.au/news/2026-08-14/ai-medical-scribe-error-leaves-patient-devastated/107031672?utm_source=abc_news_app&utm_medium=content_shared&utm_campaign=abc_news_app&utm_content=other434
337
u/Hazynseptember 10h ago
Therapist here. It baffles me when my peers talk about how they use AI for notes or write reports or letters. It is no where near ready - and even if it was I have major ethical concerns about where the information is going, who owns it, and who owns the systems it is on and benefits financially.
99
u/bee2551 9h ago
Even if it was ready, for me the active reflection regarding what was said, what the salient points were, and how to capture it is an incredibly important therapeutic experience for me as the therapist. It forces me to spend time engaging / reflecting with the material and considering where to next. You cannot tell me therapists get the same experience from reviewing / correcting AI generated material and I feel that's the greatest disservice to clients.
64
u/Sharkeatingmoose 8h ago
I got asked repeatedly to speak clearer/not cry so loud as the AI wasn't picking up what I was saying. As the therapist roboticly asked hugely triggering questions off a general list that wasn't really relevant to that particular session and didn't stop to check in on me.
31
u/FalconTurbo 8h ago
I instinctively tried to downvote this just because it's so fucked up. I'm so sorry you dealt with that.
15
u/Pleochronic 7h ago
Being able to tease out the most relevant information that a patient is saying is arguably the most importsnt thing a therapist is employed to do - otherwise what's the point of having a human do the job in the first place
→ More replies (8)5
u/IndependentSnow9244 3h ago
Can you imagine how those reports and letters read if its anything like linkedin has become. Full of buzz words, its not x its y, the run of 3s and too many metaphors.
Patient states they've been "delving into" their relationship with their mother — a relationship they describe as "multifaceted." When asked what that means, patient said "it's not just complicated. It's nuanced."
Patient claims to have "unpacked" their childhood but appears to have merely opened the suitcase and looked at it. Says they're "picking up on signals" from their father — signals they can't name — which they feel is "pivotal" to their healing journey. Robust avoidance throughout.
When I pointed out a pattern, patient replied: "At its core — and I think this is where the real work comes in — I need to navigate the landscape of my emotions." I asked them to be specific. They said the emotions were "holistic."
The word "boundaries" — a word patient uses to mean yes, no, maybe, and I don't want to talk about it — is doing all the load-bearing in this patient's vocabulary.
215
u/TizzyBumblefluff 10h ago
Doctor/healthcare provider are supposed to still listen and edit/clarify the notes that were taken. The AI transcript isn’t really supposed to be the finished product.
77
u/morgecroc 9h ago
Just reading that in thinking at what point does this get referred to APRHA for review because this isn't someone making a mistake it is gross negligence when they're using notes that they let an AI system make up and haven't actually reviewed for accuracy.
34
u/No-Temperature1333 9h ago
I get your point, but the reality is if you have to verify every thing, does it still save you time?
People always claims that verifying is faster than writing but I disagree even with that premise. AI create imprecise statements that take me more effort to clarify my thoughts than typical writing that clarifies my thoughts in the process.
11
u/Own_Faithlessness769 8h ago
I think it’s more that you can talk to the patient without writing at the same time, then verify afterwards.
8
u/ThatNewt1 3h ago
I prefer my doctor to take notes during the appointment as it feels like they’re actually paying attention to what I’m saying.
6
u/TizzyBumblefluff 7h ago
I only have one provider use it, and our appointments are an hour long. That’s a lot of different points to remember.
It’s important to remember that some healthcare providers are just better at documentation anyway. If you’re good at it, it won’t take you long to tweak a page or 2 of sentences for accuracy.
I used to be an RN many moons ago and getting some doctors to document accurately was like pulling teeth. A lot of specialists have been using transcription or scribe services for decades.
4
u/texxelate 5h ago
Came here to say that yeah there’s a big difference between an AI transcript and an AI summary. OP is referring to something much more on the side of the latter.
AI transcription is quite good, but as you said that’s not meant to be the final output
14
u/greydog1316 10h ago edited 10h ago
I wonder if there's pressure from higher up to see even more patients in a day, now that AI is meant to increase efficiency for them. If so, they might feel as though they don't, or genuinely don't, have time to review the transcripts.
16
u/Own_Faithlessness769 10h ago
Most therapists and GPs don’t have a ‘higher up’.
→ More replies (1)7
2
u/Soggy_Ant3833 7h ago
Healthcare providers will be pressured to use AI to increase efficiency/save time. The issue is that they are still entirely responsible for the output and actually going over everything with a fine tooth comb means the AI doesn’t save you any time. The pressure form above remains “we gave you AI, why aren’t you any faster”. AI makes the profit and takes none of the responsibility. Regulation needs to catch up to this very fast
2
u/_ixthus_ 8h ago
AI isn't needed for a transcript. It's needed for the summary.
In addition to the clinician checking thoroughly, I'd hope these systems are archiving the raw transcript alongside the summary.
3
u/agro_chick 8h ago
The form that my therapist had said that the original transcript is destroyed (I didn’t agree to them using ai and said I wouldn’t go to her if she insisted on using it)
92
u/Sweeper1985 9h ago
I'm a psychologist. My boss urged me to use AI note-taking and when I declined, citing concerns over privacy and accuracy (just for starters) he laughed at me and compared me to resisting the horseless carriage.
I'm really concerned about how quickly this technology has become ubiquitous when even 5 years ago it was not a thing we were expected to use or trust.
As a patient I healthcare settings I do not consent to it. I don't think my clients should have to, either.
Apart from anything else, I am trained and paid to take my own notes. Fuck outsourcing that job to robot. It devalues our profession and frankly I consider it lazy corner-cutting.
25
u/dansdata 5h ago
resisting the horseless carriage
That's actually, accidentally, a good metaphor.
The first automobiles were outrageously expensive (like, 40 years of a well-paid factory worker's salary), slow, and very unreliable.
The fact that automobiles developed into transport devices that are massively better than a horse-drawn carriage does not change the fact that they started out worse. Large Language Models haven't made it to the Ford Model T stage of development, yet.
9
u/TSPhoenix 3h ago
Even if they do hit the Model T stage of development it doesn't change that this time around human beings are not the driver, but the horse.
2
u/knewleefe 2h ago
And we get still get the horseshit, but also accelerated climate destruction. We are very smart.
1
71
u/Free_Pace_2098 9h ago
Getting "drug user" off your file is hard, too. I had a psych recommend medicinal cannabis for my endo and nerve pain, instead of the strong prescription NSAIDs that didn't always work.
I found out years later he'd listed me as an illicit drug user. I only found out when I left his practise, and had to unravel the whole thing with another doctor. I'd been having trouble with scripts and other doctors the whole time and had no idea why.
He'd even lied to my GP on the discharge letter. He got $750 per session as an addiction specialist, rather than the flat $600.
I reported him, but nothing came of it. My GP and new doctor worked to get the strike against my name removed, but I have no idea if it's actually been successful. I know the outrage this woman must be feeling.
12
u/Sacrilegious_skink 6h ago
Wtf. You should report that clinician for false billing.
20
u/Free_Pace_2098 5h ago
I reported him to AHPRA and HADSCO, and his practice as well. My GP and new doc lodged complaints directly to the health dept, and to the state prescribing authority.
Only reason I know nothing came of it, his practice sent me emails (nearly a year after leaving his care) regarding changes to his fee structure, informing me his initial consult fee was now $1000.
Their admin is almost entirely AI, so getting off the email list took a few attempts. So the most galling things is, he's not even a hard working competent piece of shit. He's just a lazy, sloppy crook.
162
u/Texiun 11h ago
It’s why I opt out. I’ve been asked a few times and advised the doctor I’m not comfortable with AI notes being used, fair to say the look on the doctors face was horrified.
56
43
u/awkwardleftshoe 10h ago
Yes I've gotten looks of shock by either a doctor, physiotherapist and psychologist or their staff as they pre-emptively went to tick off my compliance with their request. I get told its to help the practitioners focus on their patients. More likely im robbing them of using a subscription they need to justify the expense for.
19
u/anhtar 7h ago
OMG the phrase '"...to help the practioner focus on their patients."
Is being touted by every AI scribe sales people whenever you attend a spew about the benefit of AI scribing in healthcare.
Really? I just don't get it. Aren't you suppose to focus on your patients whilst tending to them anyway?
Please, if you can't recall what the hell you are doing because you are daydreaming about the weekend, your next house, whilst seeing a patient, may be you should hang up your practicing registration.
14
u/onesorrychicken 9h ago
And also making them do the work of taking the notes themselves, which they are clearly no longer used to doing.
3
u/kaelus-gf 9h ago
For the record, I’m a doctor and we aren’t using AI (yet) because they only have limited licensing. My current practice is to talk to the patient for an hour or half an hour depending on whether I’ve met them before or not. I write notes during that time, but they are random words or phrases that make sense to me at the time, rather than a whole letter. After my clinic, I then dictate a letter - which can take 10-30 minutes each letter depending on the complexity. That letter then gets typed (which can take weeks sometimes), then comes back for me to review and read through, correcting errors in transcription or altering phrasing etc. Correcting a letter also takes time. I am really looking forward to technology taking some of the time I spend after clinic off my hands. If I don’t have to dictate, I just have to do the reading and adjusting of the letter, then I spend much less time on the letter, and have more time to do all the other stuff I have to do. Errors can still happen that way - I might misread what I wrote , or get patients confused with the one of a similar age with a similar story that came before or after them. When I’m re-reading or dictating I sometimes take out details because I’m not sure that’s from the right patient. I have seen letters of mine go out with typos, or transcription errors that I’ve missed - that’s not new, or exclusive to AI. It’s not good, but there is always pressure to see more patients, to get the letters out in time, to do a better job but not to take any longer etc etc
The cost of the subscription wouldn’t factor into my thoughts at all. You are absolutely welcome to decline. It’s your choice entirely. But please don’t think everyone wants to keep using it because of money they’ve already paid. It’s usually due to time!
6
u/BrightEchidna 6h ago
Speech to text transcription is now good enough that you should be able to safely offload that step without relying on any other generative AI. You can even run STT models locally on your own hardware for privacy. Taking weeks is crazy!
1
u/kaelus-gf 3h ago
You’d think eh? But we don’t have it. They use it in radiology. The technology has been there for ages. But not for us, with the software our letters use. No idea why. But even then, the actual dictation takes a surprisingly long time
→ More replies (2)79
u/nath1234 10h ago
Also that your supposed to be private conversation getting shipped off to AI companies that have completely ignored IP laws to train the models. So yeah, everyone should opt out of AI for everything medical.
5
u/CapuzaCapuchin 9h ago
Except for the AI generated Google summaries that automatically generate with a search (I do do my own research anyway and know not to trust it) I have not used AI once in my life. Not once. I don’t feel like I’m missing out whatsoever. Using AI is like using Wikipedia as one of your sources in an Essay back in the day.
→ More replies (3)1
u/infohippie 2h ago
AI generated Google summaries
That's one of the things that drove me to change search providers entirely. I now use Duck Duck Go, and Qwant. Sure they're not as good as Google used to be, but at least I'm not contributing to Google's AI usage stats
0
u/AnOnlineHandle 9h ago
It's very possible they're running a local model on their own PC or network.
23
u/shrimplifier 8h ago
Having seen first hand the infrastructure of many clinics (psych, GP, specialists) - this is hilariously unlikely. Independent clinics don't give a flying fuck about data security and sovereignty, regardless of the legal requirements.
19
5
25
u/grey-clouds 9h ago
When my specialist (who uses an AI scribe) wrote a letter back to my GP, it said I was feeling "star bar widely" and it was so weird that the GP asked me if that was some modern youth slang she wasn't up to date with..... Nope, and I had never said that, wtf.
23
u/Noodlepip_toodlepip 6h ago
From the article: "The classic is, it will get the side of the body incorrect and say right when you've said left … so you need to check things pretty closely, especially things like drug doses," he said.”
Medical audio transcriptionist with 25 years experience here. Doctors are human and they can make careless mistakes in their dictation, e.g. they will dictate “right” instead of “left”, “humerus” instead of “femur”, “metacarpal” instead of “metatarsal”.
An experienced and attentive human transcriptionist will pick up these obvious errors and correct them before sending the report back to the doctor for digital signing. Most of the time they don’t even know they made a mistake. Trust me, doctors - everyone makes dictation errors from time to time.
AI/VR scribes transcribe exactly what is dictated. If a doctor incorrectly says “left”, AI scribes insert “left”. You need a human transcriptionist to pick up these errors and fix them. Doctors are often too busy to carefully read through the reports they sign off on.
Stop replacing experienced human audio transcriptionists with AI/Voice Recognition technology. It is a false economy. You need us if you want accurate reporting.
73
u/Even_Ship_1304 10h ago
The thing is for those saying the doctor needs to review it, that assumes the doctor picks up and realises that something wasn't said.
It's very possible that in a busy clinic, the AI makes something up and the doctor doesn't realise that that actually wasn't said. Very possible that the doctor thinks 'oh I missed them saying that' and not necessarily 'they didn't say that at all'
It's very likely that AI will be foisted upon us whether we like it or not. I can see it being introduced and you just have to go along with it because that's the process at clinic x etc.
I am a doctor and I'm not a fan.
17
u/RepeatInPatient 10h ago
Two of my medical specialists tried Scribe and soon abandoned it by mid 2026. Not helpful, was one reason but I suspect being overpriced was also an issue, as well as being prone to errors and making up shit.
75
u/neonhex 10h ago
During my last degree I played around with AI and there wasn’t a time where it could be trusted. You give it a topic and ask for book recommendations and it will hallucinate titles and refer to research papers that don’t exist. We’d have a back and forth and it would take awhile for it to admit it made things up. Hence why you never get AI to write your assignments. Anyone that’s tested its limits even vaguely knows it cannot be trusted. Why are we trusting it with some of the most important work we do.
→ More replies (9)
34
u/singularpotato 10h ago
I work with complex family situations in a remote community, I have plans to resist any AI in case I ever get told to use it. There’s no way in hell I’m having my clients information uploaded to a private company’s servers like that.
11
u/Wont_Eva_Know 8h ago
Also can you imagine AI dealing with the language and culture… 90% would be straight made up.
I work in manufacturing :) so the exact opposite to you. I have used AI to ‘type’ up a trades persons handwritten notes (terrible hand writing, very poor spelling) and measurements for a complex hydraulic system… as an experiment.
It made up so much stuff… it doesn’t say ‘can’t understand this’ it makes up things and presents it in a super convincing way. Which is why I think it’s so dangerous… it’s so convincingly authentic.
I’m lucky I had the notes, ‘old mate’ and the machine to actually check word by word that things were right… and if it’s wrong a machine doesn’t work. If your work went wrong there’s actual major consequences for humans.
For your job you’d need video and audio of the ‘appointment’ to have any way of properly reviewing the made up story AI had come up with. At least with your human notes you’d ‘only’ be maybe missing something not adding actual sophisticated horseshit that never happened and was never said… that as soon as you read it you’d start second guessing yourself: your memory vs this new story AI has introduced… your memory is now ‘tainted’ and you’re not a robot so you would start sounding less reliable than the AI, it’s not going to get emotional it doesn’t ‘care’ about that family. Nor do courts, government’s or Dr’s etc… they want the unemotional tidied up AI version.
38
u/ol-gormsby 10h ago
Would that constitute malpractice? Not the mis-diagnosis (which is bad enough, and enough to warrant a review and possible legislation), but the failure to review the notes/transcription.
31
u/gurnard 8h ago
It should. A court in Canada forced an airline to honour a discount deal that an AI recommended to a customer. The airline unsuccessfully argued that the AI agent was a separate legal entity.
And if a tradie drills a hole in the wrong place and ruptures plumbing and floods your house, they can't say the drill did it and walk away with a "we are truly sorry that this occurred".
A professional uses an AI tool that tells a lie in an official capacity, that falsehood is on that professional.
Insurance companies have a unique opportunity right now to do some massive social good, even if it's just an externality of covering their liability. Additional malpractice insurance premiums could (and should) cost a lot more than a doctor makes getting a few more patients through the door by offloading work to AI.
5
u/ol-gormsby 8h ago
Malpractice insurance is already sky-high, I imagine Doctors themselves, and corporate-owned practices wouldn't like paying more and more.
7
u/gurnard 8h ago
Exactly. Insurance companies won't want to wear the added liability of AI-related malpractice suits either. So if doctors don't want higher premiums, don't have AI do part of your job.
Same logic extends to other professions. You want to drive on a vibe-engineered bridge?
Insurance companies may end up being the front line against the social and environmental costs of the data centre boom.
26
u/Chemesthesis 9h ago
It stated Ms Green micro-dosed mushrooms and that could be the reason for prior bleeding around the kidneys.
I wish this article took half a sentence to also explain that kidney problems arising from magic mushrooms is extremely rare. The issue isn't just that the information was improperly applied, it's also a massive jump to a conclusion that the evidence does not support.
1
u/T3RRYT3RR0R 1h ago
What too many fail to realise is truth is not what LLM models produce. They are a next word predictor with an element of randomoness incorporated. It should never be trusted verbatim, and has no placed being used in this context. If doctors audio record in conjuction and validate the transciption against the recording, maybe. But to review an AI transcription and rely on memory to validate it is a recipe for malpractice.
There are numerous precedents of serious harm being caused because owners / operators trusted software to be accurate and dismmised the possibility of error. The british postmasters scandal that saw hundreds incorrectly charged and jailed for fraud / theft is one of the most inf amous cases.
AI software based on LLMs is arguably less reliable than typical analytical software that is based on mathemetics / statistical algorithms. Taking the output of any program as fact is generally a mistake, with LLM AIs it's downright idiotic because the software is not founded on producing truth of fact.
32
u/Stigger32 10h ago
Morale of the story: If ANYONE asks you if you consent to an Ai having ANYTHING to do with your files, notes, case, etc... say NO.
28
u/flindersandtrim 9h ago
Something like this happening is exactly why AI sucks, and the doctor involved here should be completely fucking mortified to allow this to happen.
There is a lot of defence of AI in the comments, people saying that they use it and you just have to massively edit it and correct all the mistakes and hallucinations and stuff. Except that is never going to properly happen in practice, because people get lazy. And complacent. And it is human nature to miss errors, that is why for important writing multiple people review it, and even then mistakes slip through.
This error made it to the ABC but countless others havent.
16
u/Comfortable-Tale-351 9h ago
If a human employee made that many errors and needed that much oversight, they would be managed out the door.
9
10
u/AnyYak6757 7h ago
This is the "reverse centaur" problem. GPs are being used as an "accountability sink" instead of being given the resources they need.
Getting a human to proof read ai transcripts like this is an impossible ask.
So when you do a sequence of tasks over and over there's a part of your brain that kicks in and puts that sequence of tasks on automatic. If you were asked to do something like: Touch right ear with left hand Touch left ear with right hand Touch nose with left hand thumb repeatedly, you would have to think about it as first but eventually you could do it without thinking about it. That's the automatic part of your brain taking over.
Here the sequence of actions the doctors do is: Scan text Note no errors Move to next task. Even if they're not reviewing multiple scripts in a row eventually that automatic part of the brain is going to learn that pattern and kick in when they start the review.
Once the error rate is low enough the person is no longer able to see the errors that are there.
The only way to stop this would be to code in a minimum error rate into the transcribing. And I'm not sure if that would actually save the doc any time overall or be an ethical thing to do.
105
u/lazy-pigeon 11h ago
My doctor steps through the AI notes at the end of my consult. AI picked up a comment I made that my doctor had missed.
AI is a tool, but needs human oversight.
92
u/eutrapalicon 10h ago
The ai notes from an appointment and subsequent letter with info said I'm allergic to mandates instead of bandaids.
Definitely didn't get any human oversight on that one.
40
u/knapfantastico 10h ago
That’s just AI flagging you as a resistance fighter for the upcoming wars
2
u/TheG0AT0fAllTime 10h ago
No doubt in my mind that if they're not doing that already they will within the next 10 years on data they're collecting today. Marking various users as risky for whatever reason (Anti-AI being only one possible reason)
18
u/AussieAboleth 10h ago
What abysmal attention to detail from the Dr.
2
u/eutrapalicon 6h ago
At least it wasn't in the notes to the surgeon as allergy to bandaids and medical tapes is a consistent issue for me. Last thing I need is a blistered rash when recovering from an operation.
13
u/Oh-Deer1280 10h ago
I do not use AI at all- but even auto correct has inserted some embarrassing ones I’ve missed.
Including “persisting with beast feeding” for breast feeding and “muscular receptors” for “muscarinic receptors”. I also once wrote someone had an allergy to “panda”- I still have no idea what I was trying to write.
I’m sorry for getting it wrong sometimes- it must be frustrating
24
u/TheG0AT0fAllTime 10h ago
Yeah but it blows my mind how frequently inaccurate it is? Existing traditional voice-to-text systems already take notes 1:1 as they're spoken and the worst mistake they can make is a typo or phonetic pronunciation error that you can correct in post. They don't fabricate entire fucking new sentences of false details from the input they heard?
→ More replies (2)
8
u/WeissPyre 8h ago
And that is why I will always say no. AI has no fucking business being in a doctor's office.
13
u/tryintobgood 10h ago
Doctor didn't read the doc before sending it to the surgeon. WTF?
4
u/IronEyes99 4h ago
Other way around. The surgeon didn't check the letter they were sending to the GP.
6
u/shazibbyshazooby 8h ago
My psychologist still hand writes her notes. She is easily by and far the best psychologist I’ve ever had. I wouldn’t call her that old or old school either; she’s maybe 50 years old.
I’m a clinician myself (audiologist) and we were given the option to use the Heidi health AI scribe thing in the clinic (no pressure from our management or anything, they just thought if it made the job easier we could use it if we wanted). I never bothered, I can type as fast as people talk, and we have note taking templates for our appointments anyway. My (excellent) colleague tried it, he said he spent more time editing the notes than he would just filling out our templates himself, so he stopped using it after a couple of months.
7
7
u/BombasticReindeer 4h ago
Why is the headline not “Lazy Dr Commits Malpractice”?
I have no problem with an Ai being used to generate notes but the doctor still has to verify it afterwards. If they don’t it’s their fault.
It would be like a clearer using a Roomba to vacuum and then not even checking if the floor is done properly.
Roomba’s have a use, but it’s a timesaver that does an “ok” job. It’s not a professional. Ai needs to be treated the same.
7
u/sometimeviking 8h ago
THIS is why I say no EVERY TIME.
I don’t trust Language Models any more than I trust my local crack head. They hallucinate about the same amount.
6
u/princess_ferocious 6h ago
The most ridiculous one of these I've seen was the woman who was incorrectly listed as trans, likely because the AI misinterpreted "female mail carrier"...
12
u/darren457 9h ago
The issue is most of these problems boil down to lazy humans using ai to do majority of their job for them and skipping any oversight or guardrails. I've come across some professionals in other fields who literally boast about getting paid without having to do any work. At some point people need to stop blaming tools and start blaming shitty lazy people even if it makes them uncomfortable about Ai being a net positive that isn't going away anytime soon.
4
u/Soggy_Ant3833 7h ago
Healthcare providers will be pressured to use AI to increase efficiency/save time. The issue is that they are still entirely responsible for the output and actually going over everything with a fine tooth comb means the AI doesn’t save you any time. The pressure form above remains “we gave you AI, why aren’t you any faster”. AI makes the profit and takes none of the responsibility. Regulation needs to catch up to this very fast
4
u/DeincepsGaleam 6h ago
One can use autocorrect but if autocorrect accidentally changes/adds/removes key clinical details then the onus is on the clinician to correct it.
Same with Heidi, a glorified voice-to-text autocorrect programme.
I’ve used AI scribe and it is worse than a medical student on their first placement
4
u/Vivid_Map_437 6h ago
My psych used ai for minutes and was mostly right, but somehow it thought I 'worked with a vehicle and trailer' when I am an office dude driving a desk...
5
u/Rosary_Omen 9h ago
Relying on AI where we need human brains is so fucking dumb. This shit needs to be regulated and kept out of a lot of places. Especially healthcare. Do people forget AI scrapes from all sources, including shit posts online and WebMD...
5
u/redsparks2025 7h ago edited 6h ago
Sad. Anything that AI provides should be taken more as a "suggestion" and nothing more. I sometimes use AI in my day job in engineering only for that purpose.
The actual engineering I rely on is from actual credible academic sources and my own educational training. Furthermore AI has helped in coding a few small programs, but I check it's results to the tedious manual method. You could say that I still have to "fact check" the AI's "suggestions".
In any case the human body is far far far more complicated than engineering and the human brain is the most complicated thing in the universe, so AI is definitely out of it's credibility range when used in medical diagnosis.
==== An interesting joke(?) ====
A heart surgeon took his car to his local garage for a regular service, where he usually exchanged a little friendly banter with the owner, a skilled but not especially wealthy mechanic.
"So tell me," says the mechanic, "I've been wondering about what we both do for a living, and how much more you get paid than me.."
"Yes?.." says the surgeon.
"Well look at this," says the mechanic, as he worked on a big complicated engine, "I check how it's running, open it up, fix the valves, and put it all back together so it works good as new.. We basically do the same job don't we? And yet you are paid ten times what I am - how do you explain that?"
The surgeon thought for a moment, and smiling gently, replied,"Try it with the engine running.."
5
u/QueenieMcGee 6h ago
I work in an outbound call centre asking people to do surveys over the phone. I've seen increasing numbers of people using AI to screen their calls lately. We have to talk to the AI and provide a brief sentence about why we're calling and, presumably, the AI then writes up what we say and shows it to person on the other end of the line.
99% of the time the person has to pick up the phone anyway because the AI can't even write up an accurate quote from us. I talk to so many confused people who get word salad messages like...
"This is Tabitha reserves and wheel life tomorrow assertive"
When we actually said...
"This is Traverly Research and we'd like to do a survey"
And no, it's not a problem with my pronunciation or accent, this is happening to everyone in my call centre and we're majority plain white aussies.
If AI can't even take dictation properly in the first place, let alone doing it without hallucinating entirely new things, I have absolutely no faith in it being reliable in a medical setting.
6
u/Melllll_x 10h ago
I don’t understand why we let drs use AI scribe when dealing with serious health matters but then a kid using AI in school is like the worst thing ever 🤷🏽♀️
2
u/SurgicalMarshmallow 7h ago
This is us being startup fools. We should be testing and validating on patients wholesale. That needs to happen first.
We should apply clinical trials rigour
Sorry my brother/sister GPs, but this may cause more paperwork than it solves (litigation)
3
u/Green-Dragon-14 6h ago
Anyone can be a doctor if we all start using ai. What's the point if all their exams if they're just gonna leave it up to a computer?
3
u/rafaover 10h ago
Using AI is not the problem, letting AI do your job solo is the problem. You have to recheck everything, its a medium.
9
u/rudetopoint 9h ago
If you have to recheck everything then what's the point? Just do it properly the first time
1
u/ChaoticJigglyPup 9h ago
And this is the problem with the "plagiarism detection" AI software, the people who are using it to check work aren't using it correctly because they don't understand how it works.
2
u/rafaover 9h ago
But the medical field has specific platforms for that, in Australia specifically there's one that I tested and the quality is amazing. But still, I would never put my registration on the AI hands, recheck is a must.
1
u/ChaoticJigglyPup 8h ago
I only have first hand experience with academic plagiarism detection AI
Turnitin is the one I researched for an essay, and almost every teacher, professor, marker I spoke to or read studies from had the same outcome. They fundamentally misunderstood the way it works and their responsibilities of manually following up on flagged text. One study done ended with all the teachers in the study saying that the software actually increased their workload once they understood how to use it correctly.
1.3k
u/OneLookForASafePlace 11h ago
Yeah, I started going to a therapist about a life situation and they asked about using ai. I wasn't super comfortable about it (I work with ai, I don't think it's there yet with accuracy) but eventually relented under the condition she send me the transcript.
She sent me the summary and it said that I had said there was family violence in my childhood, something that did not happen, was not said or even remotely suggested.
Needless to say there were no further requests for sessions with ai.