Fire In The Hole

There's a decent amount of Internet schadenfreude being aimed at Shopify CEO Tobias Lütke after a podcast interview in which he noted that while generative automation allows employees to create more, faster, when they don't then check that work, it becomes "slop grenades" (perhaps more commonly known as "workslop") lobbed at other employees.

According to a Business Insider article about the interview, "Lütke said AI is most valuable when it makes people's thinking clearer and more concise."

Okay, fair enough, but in the all-hands memo that was reported, he said that "Reflexive AI usage is now a baseline expectation at Shopify."

That's significantly different from "consider your tasks, and where generative automation can make you clearer or more concise, use it." The memo seemed to presuppose that humans simply couldn't match the clarity and conciseness of an LLM. And if people are expected to reflexively turn to generative automation to do work, and are being graded on being seen to do so the "slop grenades" are a feature, not a bug.

In the all-hands memo, it's noted that: "What we have learned so far is that using AI well is a skill that needs to be carefully learned by... using it a lot." Well, this is what everyone "using it a lot" looks like.

The problem, he said, is that the tools make it easy to pass along work that has not been properly evaluated.

From the memo that I saw reported, the message seemed to be "use the tools all the time and let the feedback mechanisms sort it out." There's no call for employees to evaluate their outputs themselves before sending them. Now, I'm going to admit that I don't know if I've seen the memo as it was sent to Shopify employees... screenshots purporting to be the message in question were shared to LinkedIn, and that's what I read.

In any event, it seems to me that Mr. Lütke is running into something that I remember encountering in my professional career... people don't do what someone tells them... they do what they hear. And a hallmark of good communication is narrowing the gap between those two things as much as can be managed. While Mr. Lütke may have intended for Shopify employees to be careful to ensure that their use of generative automation didn't simply create more work for people, when employees read that "In a company growing 20-40% year over year, you must improve by at least that much just to re-qualify. [...] That sounds daunting, but given the nature if the tools, this doesn't even sound terribly ambitious to me anymore," their focus is going to be on meeting that benchmark, especially given that the memo also noted that "We will add AI usage questions to our performance and peer review questionnaire."

Of course Shopify didn't intend for people to use generative automation willy-nilly and then simply lob the raw outputs over the wall to their colleagues. But Mr. Lütke's memo didn't make an expectation that employees would check those outputs to see if they actually provided value clear to anyone. And that's where the miss occurred. Employees internalized the implicit assumption that all automation was good automation, and left it to feedback from others to inform them if they were on the right track. But the sheer volume of material that an LLM can put out in short timeframes can make evaluating all of it difficult, especially for someone who is receiving work items from a number of people.

Perhaps the real lesson from this is that CEOs are people, too, and they're just as susceptible to believing that their words are accurate carriers of their intent as anyone else. And of finding out the hard way that they're mistaken.