The ERP went down on Saturday: what happens to the sales that came in meanwhile

In 1 minute

An ERP that goes down for a while is not unusual: there is maintenance, there are updates and there are failures. What is expensive is what happens to the sales that came in meanwhile. If every sale is saved before it is promised, retried with patience, not duplicated when repeated, and what cannot go through is set aside with its reason, the outage becomes a routine matter. If not, it becomes days of manual data entry, double orders and a close that does not reconcile.

Imagine a company that sells online and over the counter. It is fictitious, but it looks like many. One Saturday, its ERP stops answering. It could be an update that ran long, a server that restarted or a link that dropped; for what follows, it does not matter. The online store is still open. Customers keep buying. And every sale has to reach the ERP to deduct inventory, record the revenue and, later, be invoiced.

Someone in operations notices late, because a customer asks about their invoice. At that moment the company has three paths, though they are rarely taken on purpose:

1. Close the store until the ERP comes back. It is safe and it is expensive: the sale is lost and, sometimes, the customer too.
2. Keep selling and "enter it later." Someone exports the orders to a spreadsheet and later keys them in by hand. It is cheap during the outage and expensive the next day.
3. Keep selling, with every sale saved in an orderly wait, that catches up when the ERP comes back, designed not to duplicate and with the list of what could not go through.

This article is for those who run operations, finance or technology in a company whose systems depend on one another. It argues one idea: the outage is not the problem; the problem is not knowing what was left pending. And it explains, in business language, the five pieces that make the third path the normal one.

This is not an article about choosing between an online call, a queue or an event; that is covered in Online or asynchronous?. Nor is it about rebuilding the history of an order; for that there is The order is not lost; the trail is. The focus here is different: keeping, pacing and reconciling what is pending during an outage and on the way back.

Everything that follows is generic on purpose. It applies to an ERP, an invoicing system, a carrier or a partner's system: any destination that one day does not answer.

The outage is not the problem: not knowing what was left pending

No system answers all the time. Vendors publish maintenance windows, networks drop and servers restart. The web's own standard reserves a code for it: 503 indicates that the server cannot handle the request due to a temporary overload or scheduled maintenance, which will likely be alleviated after some delay [7]. In other words, the industry takes for granted that the destination will sometimes not be there.

What changes from one company to another is what happens to what arrived while the destination was away. And that is where three questions come up that, the next day, someone will ask out loud:

• How many sales came in while the ERP was down?
• How many have already arrived, how many are still waiting and how many will not go through on their own?
• Did any arrive twice?

If the answer to any of them is "we need to check," a short outage turns into days of manual work and reconciliation. And the worst part is not the time: it is that nobody can say with certainty that the list is complete.

What happens at each moment of an outage

It helps to split the outage into three moments, because something different breaks in each one.

The five pieces that follow protect what comes in during the outage; the next section covers the return and the close.

Five pieces to make an outage a routine matter

1. Save the sale before promising it

The first rule is easy to state: the sale is saved inside the company before it is sent anywhere. Not in the memory of the program that sends it, which is lost if that program restarts, but somewhere that survives a failure.

The best-known way to do it has a name in the industry: the transactional outbox. When the sale is recorded, in the same step a note is also made: "the ERP needs to be told." A separate process reads that outbox and does the sending. That way, the sale is not recorded without its pending task [3]. Note: this protects what happens inside the company; it does not confirm that the ERP received it. That confirmation comes later, when the ERP answers.

If there is messaging between the two systems, there is a complementary idea, Guaranteed Delivery: once the messaging system accepts a message, it stores it in its data store and does not delete it until the next step has received it [6].

In committee language: if the ERP does not answer, the sale waits on record instead of being lost.

2. Retry with patience and with spacing

When the sending fails, the natural thing is to retry. And that is fine, with two precautions.

Patience: each retry waits a little longer than the previous one, up to a cap, so as not to retry too much [1]. If the ERP is going to take a while to come back, there is no point asking it every second. And this applies to transient failures: if the ERP rejects a sale because of incorrectly entered data, retrying it does not fix it (that belongs to piece 5).

Spacing: the retries of many sales should not land at the same instant. Amazon's guide on retries explains it this way: if all the failed calls wait the same amount of time, when they retry they arrive together again and overload the destination again; the solution is to add some randomness to the wait to spread them out in time [1]. The same guide sums it up in a phrase worth taking to the committee: "retries are selfish"; each retry demands that the destination spend more resources to handle it, and if the destination is already overloaded, retrying makes the overload worse instead of better [1].

If the destination answers with a suggestion of how long to wait —the web standard provides a technical notice for this, called Retry-After [7]—, it is taken into account together with the agreed retry rule.

3. Stop knocking on the door

There is a point at which retrying no longer helps. If the ERP has not answered for a good while, sending it every sale so that each fails one by one only wastes resources on both sides and produces repeated failures that do not help anyone operate.

That is what the circuit breaker is for. Martin Fowler describes it like this: the call to the other system is wrapped in an object that counts the failures; when they reach a certain threshold, the circuit breaker "trips" and the following calls return with an error without the call even being attempted; and when it trips, it is good for it to alert someone [2]. Not every error should trip it, only the failures defined for that purpose [2]. Every so often it lets a controlled test through; if the destination answers again, the flow continues.

In committee language: the company finds out about the outage through an alert, not through a customer asking about their invoice.

4. Repeating must not duplicate

This is the piece that protects the most money. A retry can arrive twice: the sending went out, the ERP recorded it, but the response was lost on the way; the sender does not know whether it arrived and sends it again. Amazon's guide says it plainly: a call that fails or times out does not necessarily mean that its effect did not happen, and an operation with side effects is not safe to retry unless it is designed for it [1]. The outbox carries the same warning: the sending process may send a message more than once —for example, if it crashes right after sending it and before recording that it did—, so whoever receives it has to recognize what it already processed [3].

The way to achieve this is for every sale to travel with a unique reference number that the destination recognizes and keeps: if it arrives twice, the second one does not create a new order but returns the same answer as the first. The reference number only works if the destination really compares it, and it is tested with every system involved, including the payment one. Payment processors do it this way; Stripe, for example, saves the response to the first request under a key and, if another arrives with the same key, returns the saved response instead of repeating the operation; it also compares the data with that of the first request and keeps the key for a period of time, not indefinitely [5]. This property is called idempotency: repeating does not change the result.

In committee language: the retry is designed so that the customer has a single order and a single charge, and the reconciliation at the end checks it.

5. Set aside what does not go through, with its reason

Not everything that fails is because of the outage. Some sales will not go through even if the ERP is perfect: a product that does not exist in the ERP's catalog, incorrectly entered tax data, a price that does not match. If those sales are retried without limit, they block the others and nobody notices.

The usual practice is to set a limit on attempts and, when it is reached, move the sale to a separate queue (a dead-letter queue). The queue does not keep the reason on its own: together with the sale, the error, the attempt and who decides the next step are recorded. Amazon's queue documentation describes it like this: you set how many times a message is attempted before moving it to that queue, and setting them aside serves to isolate what was not processed and find out why [4].

What matters is not the queue: it is that it has an owner. A set-aside sale with no one responsible is a lost sale with a better name.

When the ERP comes back: order and pace

The most delicate moment is not the outage: it is the return. There are two frequent mistakes.

Sending everything at once. After a long outage, there may be hundreds of sales waiting. If they are all sent at the same time, the ERP that has just come back up receives an avalanche and may go down again. It is the same lesson as point 2, on a large scale: what is pending is sent at a pace the ERP can handle, and that pace is decided and tested beforehand, not in the middle of the emergency.

Sending out of order. If a customer bought and then cancelled, the ERP has to receive the purchase first and the cancellation afterwards. The other way round, the cancellation finds nothing to cancel and the purchase stays alive. For each sale, the order of what happens to it is defined —for example, with its reference number and a step number— along with the rule for applying it, so that the cancellation is not processed before the purchase even if there are retries. Different sales can move forward in parallel. And if a sale went through the set-aside queue, that order is respected when it is brought back; Amazon's queue documentation warns that a dead-letter queue can break the order of messages [4].

And at the end, a reconciliation: what was sent against what the ERP confirmed, with the list of what is missing. It is not distrust of the design; it is the evidence that the outage is really over.

What the customer sees meanwhile

During the outage, ideally the customer only notices what truly depends on the ERP. The useful rule is that every notice to the customer goes out once what it announces has already happened:

• "We received your order" can go out right away, because the sale is already saved inside the company.
• "Your invoice is ready" goes out only once the invoice exists, not before.
• If inventory is kept in the ERP, it is a good idea for the store to reserve what was sold on its own side meanwhile, so as not to sell the last item twice.

A notice that goes out too early says something that is not yet true, and the customer remembers it more than the delay.

The risks, in business language

Five questions for your next committee

1. The last time the ERP —or the invoicing system, or the carrier— did not answer, how many sales came in meanwhile, and how long did it take your company to find out?
2. Where are those sales stored while the destination does not answer? If the answer is "in the program that sends them," they are lost when that program restarts.
3. If a sale is sent twice, does the ERP create two orders? Ask technology to demonstrate it in testing, not to assert it.
4. Who owns the sales that did not go through on their own, and how do they find out?
5. When the ERP comes back, who decides at what pace it catches up? If nobody decided beforehand, whoever is on call decides in the middle of the emergency.

If several answers are "I don't know," take it as a signal to measure the flow before the next maintenance window.

Where to start this week

Run a drill. In a test environment, deliberately switch off the destination —the ERP, invoicing or whichever worries you most— for a period and with a volume of test sales that your team defines. Include three difficult cases: a sale whose response is lost even though the destination did record it, a sale that the destination rejects because of incorrectly entered data, and a sale that is bought and cancelled during the outage. Then answer in writing:

• How many sales came in during the drill, and how many reached the destination when it came back?
• Did any arrive twice?
• Which ones did not go through, and did anyone find out without looking for them?
• How long did the destination take to catch up, and did it handle the pace?
• Did anyone receive the outage alert, and does the final reconciliation identify each reference number as confirmed or pending?

The sheet that results is a very concrete specification of what your operation needs.

What Hábil does

At Hábil we build, to measure, the space between your systems: the store, the ERP, invoicing, carriers and partners. That is exactly where it is decided what happens to a sale when the destination does not answer.

If your diagnosis confirms that an outage turns into days of manual data entry, the proposal is in Hábil MDW: the layer that sits between the systems you already have, without replacing them. Every submission is kept with its response or its failure reason, and retries are designed not to repeat the effect. For every operation there is a record of what was sent, what each system answered, at what time and, if it failed, why. The scope, the systems that are connected and the recovery pace are reviewed with your case.

The conversation can start with the last outage you remember: how long it lasted, what was done with the sales and what it cost to catch up. With that, we review together where the risk lies in your operation. Write to us on WhatsApp; to protect your customers, do not share order numbers or personal data through that channel: we will agree in the conversation on the channel for the case.

Request your first conversation, at no cost.

Consulted on 12 November 2026. Each reference carries its confidence level: high (primary source read).