Pass It On
I was reading the most recent posting on Schneier on Security, when I found a weird, rambling conspiracy theory in the comments. The general thrust of it seems to be that "an American Citizen," who is never named, was unjustly imprisoned after the were attacked by "a Muslim" who is named on more than one occasion. It's a pretty clear attempt to slander a person, who was likely the actual victim of whatever crime occurred, by casting them as the perpetrator, and to slander the local law enforcement and judiciary, by claiming that they're in on the scheme. Oh, and there were allegations of antisemitism thrown in as a follow-on. Ho hum, nothing to see here.
But it seemed like the sort of thing that one might find posted, verbatim, in other places. After all, it had exactly zero to do with a proposed law to stop 3D printers in New York from making firearms parts, so it stood to reason that someone had taken their copypasta hatchet job on the road, and dropped the text into the comment sections of other weblogs. This is, after all, a way of spreading the message and getting it in front of more people.
So I found a snippet that came across as likely to be somewhat unique, and dropped it into Google, framing it within double quotes so the search engine would understand that I was looking for the exact string. I was somewhat surprised that it didn't seem to pop up anywhere. I was more surprised to see the generative automation overview synthesize the allegations and present them as "recent reports."
![]() |
| Names redacted, because I don't intend to help spread this inanity... |
To be sure, this is an edge case and a half... I found the results that I did because I was looking to see where else the wild allegations and conspiracy had been posted, so I'd clipped directly from the text to drop into Google, which would have really narrowed the pool of possible things that the automation would find as matches.
But that doesn't mean that it's not a problem, especially given that names are not unique identifiers of people, and the fact that the automation clearly had access to a cached or archived version of previous posts. The automation simply rolls out a list of names.
And I think that this is what people are getting at when they point out that the generative automation companies are pushing to be first and best to market, and leaving the safety aspects of things until later, or to someone else. Because this isn't a problem of "A.I. slop;" this is a matter of the automation repeating random things it finds on the web. And given that web sites are seeing less traffic, as people simply take the overviews and go, it wouldn't take much for something like this to take on a life of its own, divorced from the comments section(s) in which it was first planted.

