Why Your App Slows Down as It Grows: The N+1 Query Problem Explained

A client once showed me a reporting page that took four seconds to load with test data and eleven minutes once it had a year's worth of real orders in it. Nothing was broken. No error messages. The page just got slower every month as more data went in, until eventually someone gave up waiting and asked me to look at it.

What I found is one of the most common performance problems in business software, and one that's almost invisible until it isn't. It's called the N+1 query problem, and if you're paying for custom software or thinking about it, it's worth understanding in plain terms.

What's actually happening

Say you've got a page that lists 50 jobs, and each job needs to show the client's name. A well-built page runs one query to fetch the 50 jobs, then one query to fetch the matching client names, and joins them together in memory. Two queries, done.

A poorly built version runs one query to get the 50 jobs, then loops through each job and runs a separate query to look up that job's client. That's 1 query plus 50 more, hence "N+1". With 50 jobs it's barely noticeable. With 5,000 jobs it's 5,001 database queries to render a single page.

This happens constantly with modern frameworks because they make it very easy to write code that looks clean and works correctly, without making it obvious that it's about to hammer the database. The code passes review. It passes testing. It just falls over under real load.

Why it hides so well

The reason this problem catches so many projects out is timing. During development, test data is small: a handful of clients, a few dozen jobs, maybe some seed data someone typed in an afternoon. Everything feels instant. The page that will one day take eleven minutes takes 200 milliseconds when there are only 50 records to work with.

The problem only shows up once a system has been live for a while and real data has built up. By then it's often treated as "the software getting old" or "needing an upgrade", when actually it's one specific set of queries that were never checked against realistic volumes.

I've seen this cause genuine business pain: end of month reports that time out, dashboards that staff stop trusting because they take too long to load, hosting bills that creep up because the database is doing far more work than it needs to. None of it shows as a bug. It shows as "the system's gone a bit slow lately".

How to spot it before it bites you

You don't need to be technical to ask the right questions. If you're commissioning software or reviewing something already built, these are worth raising:

• Ask what happens to load times as the amount of data grows, not just how it performs today.
• Ask whether the developer has tested with a realistic volume of records, not just sample data.
• If a page or report already feels slower than it used to, ask specifically whether it's making more database calls than it should, rather than accepting "the database needs upgrading" as the only explanation.

A developer who knows what they're doing should be able to answer these without hesitation. If the answer is vague, that's worth noting.

How I deal with it

When I build something, I try to think about data volume from the start rather than treating it as a later optimisation. That means loading test data that looks like what the client will actually have in a year or two, not just enough to click through the demo. It also means using eager loading properly (fetching related data in one efficient batch rather than one query per record) and keeping an eye on query counts on the pages that matter most: dashboards, reports, anything that lists records with related information attached.

Because I work on my own and I'm the one writing the code from the database structure through to the interface, I don't lose this kind of detail in a handoff between people who each only see part of the system. I know why every query exists and what it's fetching, because I wrote all of it. That's not a claim that solo work is inherently better, but it does mean fewer gaps for this kind of problem to slip through unnoticed.

If you've got software that used to feel fast and now doesn't, or you're commissioning something new and want it to hold up as your data grows, it's worth having a proper look at what's actually happening under the hood rather than assuming more server power will fix it. Often it won't. A badly structured set of queries will still be slow on a faster server, just slightly less slow.

Get in touch if you want a second opinion on why something's dragging, or if you're planning something new and want it built to cope with real data volumes from day one rather than just the demo.