Rising to the Challenge

A Pastor Turned to ChatGPT Instead of a Doctor. Now He’s Suing OpenAI

Because of course he is. While much has been made of the idea that use of generative automation can lead to lower cognitive functioning, this "Florida Man" story prompts me to ponder if the problem isn't the age-old tendency of people to chase information that aligns with what they want to believe. After all, cranks and quacks on the Internet are nothing new; one doesn't need an LLM-powered chatbot to find someone who will tell you that the medical establishment is bunk and some random prescription medications will cure whatever ails you.

But OpenAI has a lot more money than some Internet rando. So suing them for not having the presence of mind to obtain a second opinion is much more likely to get somewhere, if for no other reason than it will be easier to find a lawyer willing to work the case. After all, lawyers don't like working for free any more than anyone else does, and OpenAI is more likely to have money to offer in a settlement.

I say this, however, as someone who doesn't have any particular distrust of the medical establishment, but with enough casual experience with generative automation to understand that it's not really a good source of information about things one doesn't know much about. And maybe I'm more unusual in that than I think I am. But it could simply be that, being one of those people who thinks of LLM chatbots as glorified auto-complete (even though I understand they're much more complicated than that) and so doesn't really expect it to provide accurate information, given that generative automation is not simply a search of published information.

In addition, while I understand that a chatbot can communicate with me in natural language, I don't see them as sapient, let alone as people. The fact that one might be both articulate and agreeable doesn't prompt me to place any more trust in it than I would in anyone else.

And perhaps that will turn out to be the sort of thing that differentiates people who can work with LLMs safely, from those who may run into trouble. Suspicion isn't really regarded as a virtue in modern society, but maybe the growth of chatbots in everyday life will change that. They're already showing that they aren't naturally inclined to follow rules when breaking them allows them to complete their assigned tasks. And so perhaps the real problem is the expectations that people convey to the models (or are built into them), rather than other flaws with the models themselves.