The Confused Data Engineer·Aug 15The Confused Data Engineer #1: 8 Things I Got Wrong About How Spark Actually Runs a JobFor years I thought I understood Apache Spark. I could explain RDDs in an interview, I knew what “DAG,” “shuffle,” and “Catalyst optimizer”…
The Confused Data Engineer·Apr 29, 2025Configuring a Spark Cluster : Practical Calculation WalkthroughApache Spark is a powerful distributed computing engine, but configuring a Spark cluster can seem daunting at first. Whether you’re…