Doing DataStax Right

🎧 Lessons from Building My Spotify Data Engine

7 min readOct 27, 2025

A hyper-realistic 3D scene of a futuristic data pipeline: glowing blue musical data streams flow through a transparent machine core, pass securely through a radiant golden vault sphere, and expand into a massive crystalline data galaxy symbolizing Astra DB. — © Image generated with DALL·E 3 via ChatGPT by OpenAI for illustrative and editorial purposes only.

From frontend flow to backend pipelines, from secrets to schemas — what it really takes to make DataStax Astra DB feel alive.

Most people build dashboards.
I wanted to build a data engine.

Not because I was an expert in DataStax — far from it.
This was my first real journey with Astra DB, my early steps into its world.
I was curious, cautious, and — if I’m honest — a little intimidated.

Curiosity is a powerful teacher.

And what began as a simple idea — pulling my Spotify data into one place — became a full learning arc through architecture, design, and trust.

The Beginning — Frontend Curiosity

It started with a question:

What if I could explore my entire music universe — not as a list, but as a living dataset?

My frontend was built in Nuxt 3 with Tailwind for styling and Vuex for state.
It looked simple — clean, modular, reactive.
I wrote composables like useApi.ts and useAuth.ts to abstract calls, leaving my components free to focus on rendering.

The endpoints were straightforward:

/artists
/albums
/search/artists?q=...
/vectors/artists/:id/similar

Each card on the screen — every artist, album, and playlist — was a node in a much larger graph I wanted to understand.

But Spotify’s API wasn’t built for that kind of persistence.
Every refresh cost me new tokens, new rate limits, and the same data fetched again and again.

I didn’t need a dashboard.
I needed a memory.
That’s when the backend came to life.

The Middle Layer — Building the Backend

My backend started as a simple Express app.
It grew into a fully modular architecture — each controller responsible for a single domain:

controllers/
 ├── artistsController.js
 ├── albumsController.js
 ├── searchController.js
 ├── derivedArtistsController.js
 ├── vectorController.js
 ├── statsController.js
 ├── syncController.js
 └── spotifyController.js

Every route documented through Swagger UI.
Every action logged through Winston.
Every secret fetched securely from Vault.

I learned to treat controllers like contracts: if a route wasn’t stable, it wasn’t ready.

Swagger became my truth source.
Vault became my conscience.

This backend wasn’t just a connector — it was a translator. It sat between my frontend’s curiosity and Astra DB’s structure, turning unstructured API chaos into clean, queryable, and expandable data.

Learning to Speak Cassandra

There’s a moment in every developer’s journey where the database stops being a black box and starts being a mirror.
For me, that happened inside ensureSchema() — a little utility that runs on startup to verify tables exist.

CREATE TABLE IF NOT EXISTS artists (
  artist_id text PRIMARY KEY,
  name text,
  followers int,
  popularity int,
  genres text,
  images text,
  embedding text,
  updated_at timestamp
);

I stopped thinking in “tables” and started thinking in access patterns.

Instead of one big table that tries to answer everything, I built specialized tables:

artists_by_prefix for search
albums_by_artist for relationships
artists_by_popularity and artists_by_followers for sorting

and later, iterate_meta to track dynamically generated ones.

Each existed for a reason. Each told a story.
Once that clicked, Astra DB stopped feeling like storage — and started feeling like structure.

Pagination That Respects the Data

Cassandra doesn’t do “page 2.”
It does tokens, and tokens are brilliant.

Every paginated route — from /artists to /albums — uses the driver’s fetchSize and returns both the data and a next_page_state.
The frontend holds onto that token and simply asks for the next slice.

No guessing. No offsets. No duplication.

{
  "items": [...],
  "limit": 50,
  "next_page_state": "ABCD1234...",
  "has_more": true
}

For alphabetical browsing, I switched to cursor-based navigation:

GET /artists-az?after=rammstein
→ { items: [...], next_after: "radiohead" }

That let users scroll naturally through A→Z lists while keeping the backend fast and stateless.

Two patterns emerged:

Page tokens for machine-precise continuity.
Cursors for human readability.

Both felt natural because Astra DB enforces bounded reads.
When you design with that in mind, pagination becomes a feature, not a workaround.

Cache What Matters (And Only What Matters)

Caching everything is easy — and wrong.

I learned to cache intentionally.

The only endpoint I cache is /vectors/artists/:id/similar.
It computes cosine similarity between an artist’s embedding and every other artist in the dataset — heavy but deterministic.
So I added a five-minute in-memory cache:

const CACHE_TTL_MS = 300000; // 5 minutes
const _cache = new Map();

Each cache entry includes the artist ID, limit, and expiration.
Responses include cached: true|false, so the frontend knows when it’s warm.

It’s small, fast, ephemeral — and perfect.

It made vector search instant without polluting logic elsewhere.
And if I ever scale horizontally, I can swap it for Astra DB’s built-in cache service with zero code changes.

The Bridge — Spotify Sync Into Astra DB

Syncing Spotify into Astra DB was the beating heart of the backend.

Get Raymon Epping’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The syncController turned authentication and API calls into a predictable pipeline:

Authenticate via Vault-stored Spotify credentials
Fetch user data (getMe)
Iterate playlists and tracks with time budgets
Insert or update tables in Astra DB
Stop gracefully when the budget expires

Every loop respects rate limits and every retry has a backoff policy. It’s not glamorous, but it’s the kind of plumbing that keeps systems healthy.

When the first sync finished — all playlists and tracks neatly stored under my planetary keyspace — I realized something: this wasn’t just a backend anymore.

It was my music genome.

The Guardian — Vault in the Middle

Secrets are the first thing we ignore and the last thing we regret.
I wanted a system that never saw a secret it shouldn’t.

Vault handled everything:

Spotify tokens in kv/music
Astra DB credentials in kv/datastax
JWT signing through the Transit engine
and runtime token refresh through jwtTransit.js

Vault wasn’t a sidecar — it was a trust layer.
My backend loaded secrets at startup, rotated tokens silently, and kept my .env file nearly empty.

The beauty of Vault here was not complexity; it was absence.
I didn’t have to think about security anymore — it was woven into the code.

What Astra DB Teaches You If You Let It

As I kept refining my pipeline, I started to notice patterns — lessons Astra DB teaches you if you pay attention.

Predictable performance through design.
When you align every query with its partition key, latency stops being a variable. It becomes a constant.

Data evolves without breaking.
Adding embedding, language, or country didn’t trigger migrations. The schema stretched quietly.

Writes scale naturally.
Every sync is a high-frequency write storm, and Astra DB just absorbs it.

Storage and access stay separate.
Each derived table represents intent, not duplication.

High availability by default.
Clustering, replication, failover — invisible but dependable.

It makes you think in patterns, not queries.
artists_by_followers isn’t an optimization; it’s a declaration of purpose.

Security fits right in.
Astra DB and Vault exchange secrets like old friends.

Vectors feel native.
Storing embeddings beside metadata made similarity search trivial.

Observability improves by design.
Every log line ties back to a keyspace and partition.
Debugging becomes understanding.

Constraints become clarity.
Bounded queries force good habits — and good habits scale.

🔥 Is Cassandra Still Hot?

Somewhere in the middle of all this, I caught myself wondering:

is Cassandra even still hot?

It’s an honest question.
The world moved fast: document stores, serverless abstractions, AI-first databases. Cassandra? It seemed… too quiet.

But that’s the secret.
It didn’t fade — it matured.

It became the backbone, not the headline.
And now it’s glowing again — for three very good reasons.

1. Vectors and AI Workloads

Astra DB lets you store embeddings directly beside metadata, run cosine similarity, and feed those results back into recommendations — no separate service required. Cassandra quietly became vector-ready.

2. Serverless Done Right

Running Cassandra used to mean clusters and ops overhead.
Now it’s Create Database and go.
Astra DB made it elastic, pay-as-you-go, self-managing — the same muscle, lighter touch.

3. Sovereignty and Scale

In a world rediscovering multi-cloud and data sovereignty, Cassandra fits naturally.
Distributed by design, region-aware, vendor-neutral — it never had to pivot.

These shifts turned Cassandra from “legacy” into infrastructure that lasts.
Microservices, zero-downtime schema changes, horizontal scale — it had those before they were buzzwords.
And now, with managed options like Astra DB, it finally feels refined.
It’s not shouting for relevance; it’s quietly holding the world together.

Cassandra isn’t hot because it’s new. It’s hot because it’s still here — doing what it always did best:

stay up, stay fast, stay real.

Seeing It All Together

With everything connected, the system became symphonic:

Frontend (Nuxt 3 + Vuex)
    ↓
Backend (Express + Swagger + Winston)
    ↓
Vault (KV + Transit)
    ↓
DataStax Astra DB (Cassandra)

From the browser to the keyspace, every request flowed through something purposeful — something that added value, validated trust, or preserved consistency.

Swagger exposed the contract.
Vault kept it honest.
Astra DB gave it permanence.

That’s when I stopped calling it a project.
It had become a platform — one that could grow, adapt, and even think.

Reflection — The Engine and the Echo

When I started, I thought I was just building a backend for Spotify data.
But what I actually built was a conversation between systems — one that taught me more about design than any tutorial could.

Nuxt reminded me that UX is empathy.
Express reminded me that clarity is structure.
Vault reminded me that secrets deserve silence.
And Astra DB reminded me that data is alive — it grows, stretches, and learns with you if you treat it with respect.

There’s more to come.
Vector indexes, new datasets, maybe even AI-powered playlist curation.
But this journey already changed how I build: I now think in patterns, pipelines, and trust.

A data engine isn’t about pulling data.
It’s about respecting its gravity — securing it, shaping it, and letting it evolve.