Beyond Crash Counts: Unexpected Patterns in 14 Years of NYC Collision Data
When I started this project, I expected to find a straightforward story about where crashes happen most in New York City. What I actually found was a paradox that fundamentally changed how I think about road safety data.
For my Big Data final project at NYU, I analyzed 2.25 million motor vehicle collision records from the NYPD, spanning 2012 to 2026, using Apache Spark and Hive on NYU’s Dataproc cluster. My research question was: which locations, time periods, and contributing factors are most strongly associated with severe collision outcomes, and how have these patterns evolved over time?
The paradox that started everything
The first result I ran was a simple year-over-year trend. From 2013 to 2025, total crashes in NYC dropped by 58%. On the surface, that sounds like progress. But when I looked at the fatality rate, deaths per 1,000 crashes, it had increased by 82% over the same period. By 2024, it reached 2.93, the highest value in the entire 14-year dataset.
Fewer crashes, but each one more deadly.
The structural break happened in 2020. COVID lockdowns emptied the roads. Crash volume dropped by half, but the fatality rate nearly doubled in a single year. Empty roads enabled higher speeds, and higher speeds made crashes far more lethal. Vision Zero had spent years reducing the fatality rate to a historic low of 1.00 in 2018. That progress was erased in twelve months and has not recovered since.
The borough that surprised me most
When I broke the data down by borough, I expected lower-income areas to show consistently worse outcomes, a pattern predicted by Environmental Justice research. The data was more complicated than that.
The Bronx achieved the best fatality rate of any borough in 2019: 0.49 per 1,000 crashes. That was the lowest in all of NYC that year. But post-COVID, the Bronx experienced the worst deterioration of any borough, a 571% increase by 2024. The improvements had been behavioral, not structural. When behavior changed citywide, they disappeared.
Meanwhile, Manhattan presents the opposite puzzle. It has more crashes per capita than any other borough, over 20,000 per 100,000 residents, yet the lowest average severity and zero intersections in the global danger rankings. Dense traffic forces lower speeds, and lower speeds make crashes less lethal. This is the Safety in Numbers effect, playing out clearly in the data.
The category nobody was counting
Get Yayun Zhou’s stories in your inbox
Join Medium for free to get updates from this writer.
One of my most significant findings came from a data cleaning decision. About 30% of records had a NULL value for borough. I initially assumed this was a data quality problem. After a deeper investigation, I realized it was systematic: NYPD intentionally leaves borough blank for crashes on bridges and expressways that cross borough boundaries.
Rather than dropping these 687,000 records, I classified them as INTER-BOROUGH. It turned out this category has the highest average severity and highest fatality rate of any road type, nearly twice the city street average. It had been invisible in every standard borough analysis because the data appeared to be missing.
What kills is not what’s most common
My contributing factor analysis required a non-obvious technical step. The raw data stores up to five contributing factors per crash in separate columns. Analyzing only the first column undercounts by approximately 80%. By unioning all five columns into a single factor column, I produced 1.8 million factor records for analysis.
The finding was striking. Driver Inattention is by far the most common factor, with over 562,000 occurrences. But it has relatively low severity per crash. Unsafe Speed occurs thirteen times less frequently but kills nearly as many people. Most surprising to me: Traffic Control Disregarded, running red lights and ignoring stop signs, is responsible for 331 deaths in the dataset. That is more than Alcohol Involvement, which causes 127 deaths. Yet public awareness campaigns and enforcement resources focus far more heavily on alcohol than on signal violations.
The riskiest hour is not the busiest
The temporal analysis produced another counterintuitive result. 4 pm has the most crashes of any hour of the day, with over 160,000 total. It seems like the most dangerous time to drive. But its average severity per crash is only 0.31.
4 am has only 31,000 crashes. Yet it kills more people in absolute terms than any other hour of the day. Pre-dawn hours consistently show the highest severity when crashes do result in casualties. The riskiest time to be in a crash is when the roads are emptiest.
What this means for policy
Across 2.25 million records and fourteen years, the consistent finding is that crash frequency and crash severity tell fundamentally different stories. Optimizing for crash volume will systematically misallocate safety resources toward the hours, locations, and behaviors that generate noise, rather than toward those that generate deaths.
The gap between what is common and what is deadly is exactly where the real risk lives. And it only becomes visible when you measure severity, not just counts.
Note: This project was completed as part of the CSCI-UA 476 Processing Big Data for Analytics Application course at NYU, Spring 2026. Analysis performed on NYU Dataproc using Apache Spark 3.5.3 and Apache Hive. Data source: NYC Open Data Motor Vehicle Collisions dataset (NYPD), 2012–2026. Full code available on GitHub: https://github.com/Yayun-Zhou/nyc-collision-analysis. (Author: Yayun Zhou)
