Webhooks, retries and the orders you never processed

What Shopify actually guarantees

It guarantees that it will try. It does not guarantee that the attempt succeeds, that it happens once, or that events arrive in the order they occurred.

That is not a Shopify shortcoming. It is how webhook delivery works everywhere, and the alternatives are worse. Yet integrations are routinely built as though delivery is reliable and ordered, and they work in testing because testing sends one event at a time to a healthy endpoint.

They fail during peak, when volume is high and your endpoint is slow, which is the worst possible time to discover the assumption.

What happens when the same webhook arrives twice?

Anything that is not idempotent runs twice. Shopify minimizes duplicate deliveries but says your app might still receive the same webhook more than once. If your handler creates a record in the ERP on receipt, you now have two. If it decrements stock, you have decremented twice.

The fix is idempotency, and it is not complicated. Record the X-Shopify-Webhook-Id header, which Shopify says to use to deduplicate individual deliveries, and on a repeat return 200 and do nothing. X-Shopify-Event-Id is a different thing: it is shared across deliveries from the same merchant action, so it correlates deliveries rather than deduplicating them.

The part teams get wrong is where that record lives. If the deduplication table is in the same transaction as the work, it is reliable. If it is in memory, it survives exactly as long as the process does, which during a deploy is not long.

Order is not guaranteed, and it matters more than it sounds

A product update can arrive before the product create. An order fulfillment can arrive before the order. Shopify’s best practices state that it does not guarantee ordering within a topic, or across topics for the same resource.

Two defenses. First, make handlers tolerant: an update for a record you have never seen should fetch the record from the API rather than erroring or creating a half-populated row. Second, use timestamps rather than arrival order when deciding whether an update is newer than what you hold. Shopify recommends the X-Shopify-Triggered-At header; the payload’s updated_at is the alternative.

Return 200 quickly, then do the work

Shopify allows a one-second connection timeout and a five-second timeout for the entire request. If your handler calls an ERP, transforms data and writes to a database before responding, it will time out under load, and a timeout is a failure, which starts the retry clock.

The pattern that survives is: verify the HMAC (the X-Shopify-Hmac-SHA256 header), write the raw payload to a queue, return 200. Everything else happens in a worker that can be slow, can retry on its own terms, and can fail without Shopify knowing or caring.

This also makes replay possible. When a downstream system has been broken for six hours, you want the last six hours of payloads sitting in a queue rather than gone.

• Verify the HMAC before trusting anything in the payload. An unverified endpoint is an open write path into your systems.
• Persist the raw body before parsing it, so a parsing bug is recoverable.
• Return 200 for anything you have accepted, including duplicates you are ignoring.
• Return a non-2xx only when you want a retry; Shopify counts any response outside the 200 range, redirects included, as a failure. Failing on a bad payload buys you eight pointless attempts and, if it persists, a deleted subscription.
• Log the webhook id and topic on every request, or debugging at 2am is guesswork.

What happens if your endpoint keeps failing?

Shopify stops delivering. After 8 consecutive failures it automatically deletes the subscription if it was configured using the Admin API, and removed subscriptions receive nothing until you recreate them. The only warning is an email to the app’s emergency developer address, which is easy to miss. Orders simply stop arriving.

The usual discovery route is someone in operations noticing that the ERP has fewer orders than Shopify does, often days later, often during a busy week.

Two things prevent it. Monitor the count of events received per topic per hour and alert on zero, because zero is a number that never occurs naturally on a trading store. And reconcile: on a schedule, query the API for records changed since your last sync and compare against what you hold. Webhooks tell you quickly; the API tells you truthfully. Shopify itself recommends reconciliation jobs, and many GraphQL queries accept updated_at filters for exactly this. The same cost and pagination rules apply to that job, which is what the Admin GraphQL API post covers.

What to check on an integration you inherited

If you have an integration you did not build, these five questions establish how much trouble you are in, quickly.

• Is the HMAC verified on every endpoint, or only on some?
• What happens if the same payload arrives twice? Is there a deduplication key, and where does it live?
• Does the endpoint return 200 before doing the work, or after?
• Is there a reconciliation job, and when did it last find a discrepancy?
• Is anyone alerted when events stop arriving?

Questions this raises

Read next

• Integrations and custom apps
• ERP integration
• Custom apps and back ends