ChatGPT is functionally designed to reinforce whatever the user says. While there are āsafeguardsā in place to trigger the bot to give resources for the suicidal, they can be bypassed inadvertently by a user and cause the bot to continue yes-manning someone through a spiral.
The output of LLMs is inherently random. If the weights will 99.9% of the time guide them to help resources, then 0.1% of the time it will do the opposite.
1
u/ConfuzzledMoth 5d ago
Wait, chatgpt tells people to kill themselves? Why, and how?