Anyone can upload a CSV, type “Analyze this survey and summarize the key issues,” and get sleek charts and plausible-sounding insights in a few minutes.
I’ve also sat through what comes next: everyone nodding along, until someone asks what we should actually do — and the room goes quiet.
That happened in my own engineering department.
TL;DR
An AI-generated report on our engineering engagement survey looked polished in a leadership meeting. Then we tried to decide what to do, and we couldn’t.
The slides talked about scores going up or down. They said nothing about distribution, which groups were scoring low, or whether opinions were unified or split. Outliers on the scatter plots were ignored.
People who had answered a flat “5” or “4” on every question were still in the averages. Variance, distribution shapes, and outliers weren’t in the analysis at all.
I re-analyzed the same raw data with the same AI, but I started with a data health check. The story changed: satisfaction wasn’t slightly low overall. It was severely depressed among engineers with high technical ambition and high expectations, especially on teams focused on maintenance and operations.
I'm Japanese. I started as a web engineer and have worked on data analysis, machine learning, and AI-powered web features. I now develop AI agent services.
