Member-only story
Amazon Athena: SQL on S3 Without Servers
DataForge AWS Data Engineering Series - Blog 12 of 20
The analytics team’s Athena workgroup had no query result limit and no per-query data scan cap. One analyst, testing a new dashboard, accidentally ran a SELECT * with no WHERE clause against a 40TB table — a table that existed specifically because it hadn't been cleaned up in over a year. The query itself failed after several minutes with a size limit error, but not before scanning enough data to generate a $200 charge for a single query that produced no usable output at all.
Nobody had set a per-query scan limit on the workgroup, because nobody had considered that a query could be run without a WHERE clause against a table that large. The fix was a five-minute configuration change — a workgroup-level data scan limit — that would have turned a $200 mistake into an immediately-rejected query with zero cost. The gap wasn’t technical difficulty. It was that Athena’s cost-control features are opt-in, and a team that hasn’t been burned yet usually hasn’t opted in.
Blog 7 introduced Athena as part of the compute decision framework. This article goes deep on making Athena actually safe and efficient to operate — the partitioning, format, and workgroup configuration that determine whether Athena is the cheapest service in your AWS data stack or the one with the least predictable bill.
Let’s go deep.
