App performance budgets: making vendors prove it

Performance budget conversations usually stall on the question of what the number should be. On Shopify, that question is already answered, and it is answered by the platform the vendor sells through.

Shopify's app requirements state that an app "shouldn't reduce Lighthouse performance scores by more than 10 points". The test method is published with it: Lighthouse scores before and after installation, across three page types weighted home 17%, product details 40% and collection 43%. The same 10-point threshold applies to Built for Shopify status, so a vendor advertising that badge has explicitly signed up to it.

Two things follow. The first is that you have a measurement protocol you did not have to design — the same three pages, the same before-and-after, the same weighting. The second is rhetorical and more useful in practice: when you ask a vendor to demonstrate their app's impact, you are not imposing a bespoke hurdle. You are asking them to repeat a test they already passed, on your store rather than a demo one.

The asset guidance underneath it is equally quotable. Shopify recommends a minified JavaScript bundle of "16 KB or less", an entry point of "less than 10KB of JavaScript and less than 50KB of CSS on a page", loading non-critical resources on interaction, and using defer where execution order matters and async where it does not. If you want a technical question that separates vendors quickly, ask for their entry-point size against that 10 KB figure.

The temptation is to buy a monitoring tool first. Do that later, if at all. The test that answers the question you actually have is the one Shopify already specifies, and you can run it on a preview theme in an afternoon.

Take a theme preview without the app, and the same theme with it installed and configured as you would actually use it. Run Lighthouse against the home page, a real product page and your largest collection page, several times each in the same conditions, and compare the averages using Shopify's weighting. Shopify's own caution applies: "Lighthouse scores can vary between runs. Consider running these tests frequently during your development, and averaging your scores across a few consecutive Lighthouse tests." One pair of numbers is an anecdote.

Lighthouse is a laboratory test, so treat it as the pre-purchase gate rather than the whole picture. Once the app is live, the question changes to what real visitors experience, and field data on the Core Web Vitals is what answers that. The two are complementary: lab data tells you whether to install it, field data tells you whether to keep it.

Two details that catch people out. Configure the app the way you will really run it — an app measured with every feature switched off is a measurement of nothing. And check the collection page hardest, since Shopify weights it most heavily and it is where an app that injects per-product markup does the most damage.

A budget that lives in an engineer's notes is a preference. A budget that lives in the procurement process is a constraint, and the difference is whether anyone is allowed to act on a failure.

The cheapest place to enforce it is the trial. Every app worth having has one, and the trial is the only period in which removing the app costs you nothing and embarrasses nobody. Run the test in week one, not week four. An app that has been in production for a month has acquired a champion in your business, and the conversation stops being about milliseconds.

For anything significant, ask for the numbers in writing before you install: entry-point JavaScript and CSS sizes, which pages the app loads on, whether it can be limited to specific templates, and what it does when their API is slow or down. A vendor who has done the work answers these in a paragraph. A vendor who cannot is telling you something useful.

Then write the standard down where the next person can find it: apps are tested during trial against Shopify's 10-point threshold, re-tested quarterly, and a named person can remove one that fails. That last clause is the one that matters — without it the test produces a report nobody acts on, which is worse than not testing, because now you can prove you knew.

Most performance governance fails because it is too heavy. The minimum viable version is one recurring calendar entry per quarter and one spreadsheet: app name, owner, what it is for, the measured weighted Lighthouse delta, and the date last tested. Five columns.

Run it against the three page types, average a few runs, and look at the total as well as the individual rows. If the whole stack is costing you more than the sum you would tolerate from any one app, the problem is the number of apps, not the worst one — and that is a different project. Untangling it is most of what a [speed optimisation](/services/optimize/shopify-speed-optimization) engagement does, and the spreadsheet is how you know it is needed before the traffic tells you.

The honest note to end on: a performance budget is a decision-making tool, not a moral position. Its job is to make the trade explicit — this app costs this much speed and delivers this much value — so the answer is a choice somebody made rather than an accumulation nobody noticed.

• Shopify speed optimisation
• Core Web Vitals after every app you installed
• Theme app extensions: apps that do not touch your code