Your Software Passed Testing. Why Did It Fall Over on Launch Day?

I've had this conversation a few times now. A client tells me the previous system "worked fine in testing" and then fell over the first morning real customers used it. Nobody lied about the testing. It genuinely passed. It just wasn't tested against anything like the real world.

Performance testing is one of those things that sounds like a box to tick. Run the app, click around, make sure nothing crashes, ship it. But that kind of testing tells you almost nothing about what happens when fifty people hit the same booking form at once, or when a database has three years of records in it instead of the twelve rows someone typed in during development.

What "it worked in testing" usually means

Most testing happens against a small, clean dataset, with one person clicking through the app at a time. That's fine for checking the logic is right, the form validates properly, the email actually sends. It tells you almost nothing about speed or stability under real conditions.

The problems that actually take a system down tend to be invisible at that scale:

• A page that queries the database once per row on screen, fine with ten test rows, unbearable with ten thousand real ones
• A booking flow that works perfectly for one person, but creates a duplicate booking when two people click "confirm" within the same second
• A third-party API call that responds instantly in a sandbox, but rate-limits you the moment real traffic arrives
• A report that runs in under a second on a fresh database, then takes forty seconds once a year of orders has piled up

None of these show up if you test with clean data and a single user. They only show up when the conditions look something like launch day.

What realistic testing actually checks

Realistic testing isn't about fancier tools. It's about asking better questions before you write a single test.

What does the data actually look like a year from now?

If the system will hold thousands of customer records, orders, or bookings, test it with thousands of rows, not a handful. I've written before about how queries that look fine on a small dataset can quietly multiply as it grows. If you haven't seen that pattern before, it's worth reading about the N+1 query problem, it's one of the most common causes of a system that gets slower every month for no obvious reason.

What happens when more than one person acts at the same time?

Real usage isn't sequential. If your business has a busy period, whether that's a 9am rush of bookings or a Black Friday spike in orders, you need to know what happens when several requests hit the same record simultaneously. Does stock get oversold? Does a payment get taken twice? Testing one action at a time will never catch this. You need to deliberately fire concurrent requests and see what breaks.

What happens when a dependency is slow, not just down?

Most people test what happens if a payment gateway or shipping API is completely offline. Fewer test what happens if it just responds slowly, five, ten, twenty seconds. A slow dependency can be worse than a dead one, because requests pile up waiting for a response instead of failing fast, and the whole system can grind to a halt behind it. This matters even more once slow work is happening in the background rather than inside the request itself, which is exactly why I move things like emails, PDF generation and API calls into queues rather than the request, so a slow third party doesn't take the whole page down with it.

What does the busiest realistic day look like, not the average day?

Average load tells you almost nothing useful. What matters is the peak: the Monday morning everyone logs in at once, the campaign that drives a spike in sign-ups, the end of the tax year. If you only ever test against typical traffic, you'll find out about your actual ceiling on the one day you can least afford to.

Why this matters more for integrations

This gets sharper when a system doesn't just talk to itself but to other people's APIs. An integration that's tested by calling an API once, seeing a green tick, and moving on will tell you nothing about what happens when that API is rate-limited, returns data in a slightly different shape under load, or takes twice as long to respond during its own busy period. Testing an integration properly means asking what happens at volume, not just whether it works once.

What I actually do about this

On projects with real payment flows, booking volume, or busy third-party integrations, I build in tests that mimic realistic conditions before launch, not just the happy path with clean data. That means loading in data at something close to real scale, firing concurrent requests at the parts of the system where two people acting at once is a genuine risk, and checking what happens when a dependency is slow rather than assuming it's either fully up or fully down.

It's not glamorous work and it rarely shows up as a feature anyone notices. But it's the difference between a launch day that goes quietly and one where you're refreshing error logs at 9am wondering why the booking form has stopped working. If you're about to launch something that a lot of people will hit at once, it's worth asking whoever built it exactly what "tested" meant, not just whether it happened.