Let’s give a fictional shipping integration a perfectly reasonable morning: 600 labels to create, one provider, and a sample allowance of 60 requests per minute. These are illustrative numbers, not a carrier’s published quota.
Even if every request succeeds, that workload needs at least ten minutes at the stated rate. Adding twenty workers will not shorten the provider’s minute. It will mostly arrange a more energetic disagreement with it.
API rate limits need to influence how work is accepted, scheduled and recovered. The limit belongs in the design before the backlog belongs to somebody’s afternoon.
Follow one label through the queue
Accepted does not mean printed
The application records a label request durably and gives it a status such as queued. A background worker sends it when capacity is available. The user can leave the screen without the job disappearing.
That status needs to be honest. A button saying “Create labels” should not immediately produce 600 green ticks because the requests entered a queue. Green ticks are remarkably cheap when they don’t have to represent anything.
Show completed, waiting and failed items separately. If fulfillment has a cutoff, show the oldest waiting job and an estimated delay, clearly marked as an estimate. Staff need to know whether to wait or intervene.
Our integrations and AI team builds connections that keep records in step.
Control the shared allowance
A provider may apply limits per account, credential, endpoint or another boundary. Multiple workers using the same account need to respect the same allowance. Otherwise, each worker can follow its own polite little limit while the group behaves like a school outing at the biscuit table.
Also distinguish request rate from concurrency. Sixty requests a minute and five requests in flight constrain different things. Slow responses can exhaust connections even when the rate looks acceptable.
ASP.NET Core’s incoming rate-limiting middleware can protect your own endpoints. It does not automatically coordinate outbound shipping requests across your workers. That requires a separate scheduling policy matched to the provider.
Let failures return with manners
A throttled response should lead to the provider’s documented recovery behavior, including Retry-After when supplied and applicable. A temporary network error may justify another attempt. Invalid address data usually needs correction instead.
Keep the retry policy explicit:
- Limit the attempts and total time spent on one job.
- Increase delays between transient failures and add jitter so workers do not return together.
- Reuse a stable operation identifier where the API supports idempotency.
- Check uncertain outcomes before creating another chargeable label.
- Send exhausted jobs to a visible recovery path with enough context to investigate.
A timeout does not prove the provider did nothing. It may have created the label before your connection gave up. This is where blind persistence becomes expensive stationery.
Work out whether the queue can recover
Batching can reduce overhead when the API offers a suitable batch operation, but verify how the provider counts its cost. One HTTP request is not necessarily one unit of quota.
Next compare incoming work with sustainable processing capacity. If 80 new jobs arrive every minute and only 60 can leave, the backlog grows even on an entirely healthy day. No retry strategy fixes that arithmetic. You need less demand, a different allowance, a different integration path or a revised promise.
If this was useful, try The integration works. Until someone presses the button twice.
Give urgent work a deliberate policy, too. A large historical import should not silently consume every slot needed for today’s dispatch. Reserve capacity or separate workloads where the provider’s constraints allow it.
I also want a recovery rehearsal: pause the provider, let requests accumulate, restore it and watch what happens. The queue should drain at a controlled pace while new work continues. If every waiting job immediately retries, we’ve built a starter pistol.
Record the provider’s limits, retry rules and escalation route alongside the integration. The next person investigating a delay should not have to reverse-engineer yesterday’s optimism.
A healthy queue buys the business time. It cannot manufacture capacity. Once that distinction is visible on the screen, staff can make a decision instead of pressing the button again and hoping the internet respects determination.