Trial and error

Customer service bots that can’t answer what seem like basic questions. Executives who mandated AI everywhere and now own the cost overruns. Lawyers who got caught citing cases that don’t exist.

These stories are often treated as evidence that AI is overhyped and doomed to fail, maybe even that it should be discarded.

They’re better understood as evidence of learning. Of trial and error.

I’ve spent the past year figuring out how AI can help me. At various times, I’ve worked with AI to plan fitness regimens, shape marketing plans, and create elaborate automations on my phone to send morning briefings with the news and my to-do list.

Some of these uses I abandoned within a week. Some were almost right but required revision. And some, like using AI to help explore ideas, have worked from the start.

My misses were the only way to find out what worked for me. I couldn’t predict in advance where a tool would or wouldn’t fit my needs.

Likewise, we can’t derive from first principles how to use AI in customer service, or in management, or in law. The stakes might be larger for a business than for me, but the trial-and-error approach to learning is the same. Christopher Mims’ recent Wall Street Journal piece makes a complementary point: trial and error is how we learn where not to use AI.

We have to try things and watch what breaks.