Building a Real-Time GenAI Sentiment Engine for Your Stock Watchlist
How I redesigned a stock-watching pipeline around an LLM instead of a rules engine — and what broke along the way
If you’ve ever managed a personal or professional stock watchlist, you know the real problem isn’t a lack of information — it’s the opposite. Dozens of news sources, press releases, earnings call transcripts, and social chatter hit the wire every minute for even a modest portfolio of 20–30 tickers. No human can read all of it, let alone react to it in time to matter.
The traditional answer to this has been keyword-based alerting: if the headline contains “downgrade” or “lawsuit,” fire a notification. It’s brittle, noisy, and blind to nuance — “Company X cuts losses” and “Company X cuts guidance” trip the same keyword filter but mean opposite things for a portfolio.
This is exactly the gap generative AI closes well. Modern LLMs don’t need hand-tuned keyword rules; they understand context, sarcasm, hedging language, and financial framing well enough to classify sentiment with a level of nuance that used to require a dedicated NLP team. The architecture below is one I’ve been iterating on: a real-time pipeline that ingests news, scores sentiment with an LLM, and drives automated watchlist notifications.
The Architecture, End to End
At a high level, the pipeline looks like this:
Figure 1 — Reference architecture: real-time ingestion to LLM-scored sentiment, served to the watchlist UI.
Each stage exists to solve a specific problem, and it’s worth walking through why.
1. Ingestion — News Sources & External Feeds
Multiple independent feeds (financial news APIs, RSS feeds, press release aggregators, filings) publish raw events. Deliberately keeping these as separate producers rather than one monolithic scraper means each source can fail, rate-limit, or change its schema independently without taking down the whole pipeline. Each producer’s only job is to normalize its payload (ticker, headline, body, timestamp, source) and publish it — no analysis happens here.
2. The Backbone — Kafka Message Queue
This is the component that makes the whole system resilient. A single stock-news-events topic decouples ingestion from processing. A few reasons Kafka specifically earns its place here rather than a simpler queue or direct REST calls:
- Buffering against LLM latency. LLM inference is slow relative to a news firehose. Kafka lets news arrive in bursts (e.g., during earnings season) without dropping events while the consumer catches up.
- Replayability. If you change your prompt or swap models, you can reprocess historical events from the topic’s retention window instead of losing them.
- Multiple consumer groups. Today the only consumer is the sentiment pipeline, but the same topic can later feed a separate anomaly-detection service or an analytics warehouse without touching the producers.
3. Backend Consumer
A stateless service subscribed to the topic. Its job is intentionally narrow: deserialize the message, do lightweight validation and deduplication (the same story often gets published by three sources within minutes of each other), and hand it off to the sentiment stage. Keeping this layer “dumb” makes it horizontally scalable — you can add consumer instances to increase throughput without any coordination logic beyond Kafka’s own consumer-group partitioning.
4. The Brain — Gen AI LLM Sentiment Analysis
This is the stage that replaces what used to be a bag-of-words classifier or a fine-tuned BERT model. Rather than binary keyword matching, the LLM is prompted to reason about the financial implication of the text for the specific ticker mentioned, not just its surface tone. A simplified version of the prompt structure:
System: You are a financial sentiment classifier. Given a news
headline and body about a specific stock ticker, classify the
likely short-term market sentiment as POSITIVE, NEUTRAL, or
NEGATIVE. Consider financial implication, not just tone. Respond
with only the label and a one-sentence justification.User:
Ticker: ACME
Headline: "ACME cuts full-year guidance despite record quarterly revenue"
Body: {article text}The nuance matters here: a naive classifier sees “record revenue” and calls it positive. The LLM correctly weighs “cuts guidance” as the dominant signal for forward-looking sentiment — this is the exact class of error that keyword systems get wrong constantly.
Structured output (JSON with sentiment, confidence, rationale, ticker) makes this stage easy to consume downstream and easy to audit later — you always have a human-readable reason attached to every automated decision, which matters enormously when the output is about to trigger a trade-adjacent notification.
5. Persistence — MongoDB
Sentiment records are naturally document-shaped and schema-flexible (extra fields like model version, confidence score, or source metadata get added over time), which is where MongoDB fits better than a rigid relational schema. A representative record:
{
"ticker": "ACME",
"headline": "ACME cuts full-year guidance despite record quarterly revenue",
"sentiment": "NEGATIVE",
"confidence": 0.82,
"rationale": "Guidance cut outweighs the revenue beat for forward outlook.",
"source": "external-feed-2",
"model": "sentiment-llm-v2",
"timestamp": "2026-07-27T14:32:00Z"
}Storing every scored event (not just the latest) also gives you a sentiment time series per ticker — which turns out to be more useful than any single snapshot, since a stock trending from neutral to negative over six hours is a very different signal than one that’s been negative and flat all day.
Get Sumeet Singh’s stories in your inbox
Join Medium for free to get updates from this writer.
Tracing a single event through all six stages makes the latency profile of the system obvious:
Figure 2 — A single event’s round trip. The LLM inference call is, by a wide margin, the slowest hop in the pipeline — everything else is sub-second.
That gap between the consumer’s request and the LLM’s response is the one place this architecture is genuinely bottlenecked, and it’s worth designing around rather than fighting: batch smaller requests where you can, and lean on Kafka’s buffering (mentioned above) to absorb bursts rather than trying to make the model respond faster than it reliably can.
6. Watchlist User Interface
The UI reads aggregated sentiment per ticker and renders it against the user’s watchlist and positions — and this is where the pipeline earns its keep. Rather than a user having to poll the app, the system can push notifications the moment sentiment shifts meaningfully for a ticker they hold or watch.
From Sentiment to Action: Automated Notifications
The real payoff of this architecture isn’t the classification itself — it’s what you do with it. A few patterns worth calling out:
- Delta-based alerting, not threshold-based. Notifying on every “NEGATIVE” event gets noisy fast, especially for volatile tickers that oscillate between neutral and negative on unremarkable news. Alerting on sentiment shifts (e.g., three consecutive negative reads after a stretch of neutral) produces a far higher signal-to-noise ratio.
- Position-aware weighting. A negative sentiment event on a stock you hold a large position in deserves a push notification; the same event on a watchlist-only ticker might just update a dashboard badge.
- Confidence gating. Low-confidence LLM outputs (the model itself can express uncertainty) are logged but suppressed from triggering notifications, which cuts down on false alarms from ambiguous articles.
Put together, the decision logic that sits between “a sentiment score exists” and “the user’s phone buzzes” looks like this:
Figure 3 — Every new sentiment record passes through three gates before it can reach the user, and only a held position gets a high-priority push.
The last branch — active position vs. watchlist-only — is the one that does the most to keep the system trustworthy. It’s tempting to notify on every sentiment shift for every ticker a user is tracking, but that’s how you train people to ignore your alerts within a week.
Design Trade-offs Worth Naming
No architecture is free of compromises, and it’s worth being honest about them:
- LLM cost and latency vs. a fine-tuned classifier. A dedicated model like FinBERT is cheaper and faster per inference, but far less flexible when you want to change what “sentiment” means (e.g., incorporating sector context or portfolio-specific risk). The LLM approach trades raw throughput for reasoning flexibility — Kafka’s buffering is what makes that trade-off tolerable.
- Hallucination risk. An LLM can misattribute sentiment to the wrong ticker in multi-company articles, or over-index on a single alarming word. Constraining the prompt to a single explicit ticker and requiring a justification string in the output makes these errors auditable rather than silent.
- MongoDB vs. a time-series database. Mongo is a pragmatic choice for flexibility, but if sentiment volume grows large, a purpose-built time-series store (or a Mongo time-series collection) becomes worth the added operational complexity for query performance on trend analysis.
Where This Goes Next
A few directions this architecture naturally extends toward:
- Multi-agent verification — a second LLM pass that cross-checks the first classification before it’s persisted, catching disagreement cases for human review.
- Retrieval-augmented context — feeding the model recent price action or prior sentiment history alongside the article, so it’s not classifying each headline in isolation.
- Backtesting the signal — before trusting sentiment-driven notifications with real capital decisions, running the historical sentiment stream against realized price movement to validate that the signal actually correlates with anything actionable.
Closing Thoughts
The shift from keyword rules to LLM-based reasoning doesn’t just make sentiment analysis more accurate — it changes what’s even possible to ask the system. “Is this negative?” becomes “why is this negative, and does it matter for this specific position?” That’s a much more useful question, and it’s the one this architecture is built to answer.
If you’re building something similar, I’d genuinely like to hear how you’re handling deduplication across news sources or confidence-gating on notifications — those were the two areas that took the most iteration to get right.
