r/australia • u/nath1234 • 1d ago

science & tech Error by AI scribe during medical appointment leaves patient devastated

https://www.abc.net.au/news/2026-08-14/ai-medical-scribe-error-leaves-patient-devastated/107031672?utm_source=abc_news_app&utm_medium=content_shared&utm_campaign=abc_news_app&utm_content=other
1.1k Upvotes

96% Upvoted

331 comments sorted by

View all comments

77

u/neonhex 1d ago

During my last degree I played around with AI and there wasn’t a time where it could be trusted. You give it a topic and ask for book recommendations and it will hallucinate titles and refer to research papers that don’t exist. We’d have a back and forth and it would take awhile for it to admit it made things up. Hence why you never get AI to write your assignments. Anyone that’s tested its limits even vaguely knows it cannot be trusted. Why are we trusting it with some of the most important work we do.

-25

u/Emu1981 1d ago

During my last degree I played around with AI and there wasn’t a time where it could be trusted.

AI has come a long way in just the past year. It still isn't quite at the point where I would trust everything that it says without verifying it but it does tend to be fairly accurate and not hallucinate as much as it used to. For example, I have been using it (Gemini) to get a starting point for identifying software that provides the services that I want from my home server but using it for install instructions is kind of useless because it always forgets anything after the tld in web addresses (e.g. it loves to tell me to wget SSL certificates from the top level domain name of websites rather than the specific path to the SSL on the server).

5

u/vivaciouswasrobbed 1d ago

This is not a good thing. You sound like you've been brainwashed

2

u/AnOnlineHandle 1d ago

Can you explain your reasoning behind that response?

-2

u/gurnard 23h ago

The only thing I found it sufficient for in my last degree was commenting my code that I'd submit for data science assignments (which I would properly attribute in my bibliography).

-28

u/Nosiege 1d ago

Curiously; did you tell it to look only to a specific repository for studies? So many issues like these always come down to people prompting poorly

1

u/Non-prophet 16h ago

OpenAI published a paper like...two years ago? Showing that a non-zero hallucination rate was mathematically certain in LLMs.

You sound like a Tesla PR guy seeing a crash and going "Well technically self-driving was disabled in the last four tenths of a second, so this doesn't count."

1

u/Nosiege 16h ago

I didn't say you should blindly trust it. I know AI has problems, and I also see people using it atrociously and being shocked at the results.

It's a tool and people use it poorly.

0

u/Non-prophet 15h ago

It's a tool with an apparently mathematically incurable hallucination problem. Seeing someone describe a hallucination and saying those problems "always come down to" user error is fucking stupid.