This article was originally published on Medium
The Problem I Was Trying to Solve
My task was to build a service that retrieves information from different websites and organizes it. The service collected content from five different sources, and each source gave me about 120 news headlines, which is around 600 headlines total that needed to be sorted.
My first idea was simple and quick in my head: send all 600 headlines to Gemini AI and let it read through everything to figure out which headlines were talking about the same stories and which ones were completely different. This seemed like it would work well since AI is good at understanding language and reasoning.
Why My First Approach Didn’t Work
Unfortunately, this plan ran into some serious problems:
Context Length was too high: Sending 600 headlines to Gemini all at once was overwhelming for the free version and resulted in a very high context length. When you give AI too much text to process, it actually gets worse at giving you accurate answers.
Calculated Cost was too high: Even with a paid Gemini API, processing this much data would cost a lot of money every single time the service ran. This wouldn’t be affordable for regular use and would result in very high costs.
High Latency and Slow Response: Sending all that data to an external AI service and waiting for a response took too long. Users would be waiting around for results that ca be almost instant.
I realized that using AI to read and compare hundreds of headlines wasn’t practical. I needed a completely different solution.
My New Approach
Recently, I have been working with a lot of RAG techniques, and that gave me an idea for a better way to solve this problem.
Here’s what I did instead:
- Converted Text into Embeddings: I used the Gemini Embeddings model to convert each title into 765-dimensional vectors.
- Stored them in a Vector Database: I stored all the embeddings in the vector database. Supabase comes with a PGVECTOR extension, which makes this easy and efficient.
- Use Cosine Similarity on Headlines: Finally, I used cosine similarity on one of the titles to find similar titles in the vector database.
This new method had big advantages:
- Much Faster: Comparing vectors is faster than having AI read and understand text.
- Much Cheaper: An Embedding model is much cheaper than using a reasoning model.
- Consistent Results: The system provides reliable results every time as it works with accurate data and numbers, performing the same calculation each time, whereas an AI model gives different outputs every time.
- Easy to Scale: If I need to handle more headlines in the future, the system can handle it without becoming slow or expensive.
The new system correctly grouped about 90% of similar headlines together; however, can be made more accurate with a combination of both semantic search and keyword search while also comparing categories of titles.
What I Learned
This experience taught me some important lessons about solving technical problems:
Use the simplest solution that works: Just because AI is powerful doesn’t mean it’s the best tool for every job. Sometimes simpler methods work better and cost less.
Think about growth from the beginning: Something that works with a small amount of data might completely fall apart when you have more data. It’s important to plan ahead.
Learn from other solutions: The technique I used wasn’t originally designed for grouping headlines, but I realized the same concept could work for my problem, too.
By switching from an AI-based approach to a number-comparison approach, I turned an expensive and slow feature into something fast, affordable, and reliable. This showed me that understanding your specific problem and choosing the right tool matters more than just using the most advanced technology available.
If you’re building an AI-powered SaaS or content-heavy product and want an engineering team that can scale aggregation without burning LLM budget, we can help. Book a free SaaS plan and quote here