A Shopify store speed test answers one of four questions. Which one did yours?
Four different things get called a Shopify store speed test, and they answer four different questions. Which one your "we tested it" actually was — and what it could not tell you.
Someone tells you the store is slow. You run a Shopify store speed test. It comes back good. You move on.
That sequence is where most Shopify speed work ends, and there is a specific reason it can be wrong — not because the tool lied, but because four different things get called a store speed test, and they answer four different questions.
Here is the separation.
| The test | What it actually answers |
|---|---|
| Lighthouse / PageSpeed lab run | Is the page fast under one controlled set of conditions? |
| Field data (real-user) | Is it fast for the people who actually visit? |
| Desktop browser resized narrow | Does the layout adapt to a smaller screen? |
| A real phone in your hand | Can a customer actually complete the purchase? |
2.5 seconds is a distribution, not a reading
The number everyone quotes is right: 2.5 seconds for Largest Contentful Paint.
What the number means is where it goes wrong. It is not “your test should say under 2.5s.” It is a percentile over real sessions.
A percentile over real sessions, not a reading from one run. Which produces a situation worth naming plainly:
- You test the store yourself → 1.8s. Good.
- Real customers, on their own phones and networks → some at 3s, some at 4s, some at 5s.
- Enough of them above 2.5s and the store fails the threshold your test just passed.
The particularly bad version is the one that feels like success: Lighthouse says “Good” and your customers are waiting. A lab run is a single device on a single connection. It cannot tell you about the spread, and the threshold is defined on the spread.
| Criterion | A lab run (Lighthouse / PageSpeed) | Field data (Search Console / CrUX) |
|---|---|---|
| What it measures | One page load, one simulated device, one simulated connection | Real sessions from real visitors on their own devices and networks |
| The question it closes | Which resource on this page is slow? | Is this page slow for the people who actually arrive? |
| Can it evaluate the 2.5s threshold | No — the threshold is a percentile and this is one sample | Yes — that is what the metric is defined on |
| Available on a low-traffic store | Always | Often not — there may be too few sessions to report |
The honest version: The lab run is the better diagnostic, and it is not close. Field data tells you that a page is slow; it will not tell you which image, script or app is responsible. When you are deciding what to change, the lab run is the instrument you want — and on a store with too little traffic to report field data, it is the only instrument you have, and optimising against it is correct.
The practical consequence: a passing lab score is not evidence the store is fast for customers. It is evidence the store can be fast. Those are different claims, and only one of them is about your customers.
A narrow desktop window is not a phone
Drag your browser window narrow and you have changed one variable: width.
That tells you something real — whether the layout reflows. It is a layout test, and it answers a layout question.
What it does not reproduce, on an actual handset:
Only a real device will show you these
- A sticky add-to-cart bar sitting over another control. The bar is fine at desktop width because it often is not rendered there at all.
- Tap targets close enough that a thumb hits the wrong one. A cursor is one pixel; a thumb is not.
- Mobile Safari and mobile Chrome behaving differently from each other.
- The on-screen keyboard taking a third of the viewport the moment a field is focused — and a form that scrolls underneath it.
- Sticky and fixed elements interacting with the browser's own collapsing toolbar.
- The in-app browser: a store opened from a link inside Instagram, Facebook or TikTok is not running in the customer's browser. It runs in the app's, and it does not always behave the same.
That last one is worth sitting with, because of where the traffic comes from. If you are buying paid social, a meaningful share of your visitors never touch their own browser — they arrive inside the app that served the ad. Testing in Chrome on a laptop tests a visit that cohort never makes.
So which Shopify store speed test should you run?
All four, and knowing which question each one closes.
-
Field data first, if you have it
Search Console's Core Web Vitals report, or CrUX. This is the only one that describes your customers. If it is empty, you do not have the traffic to report, and that is your answer for now.
-
A lab run, to find out what is slow
Field data tells you that it is slow; Lighthouse tells you which resource. The lab run is a diagnostic instrument, and it is good at that job.
-
Resize the window, for layout reflow
Fast, and it finds a real class of problem. It answers a layout question and should not be asked to answer any other.
-
A real phone, on cellular, including one opened from an Instagram link
Add to cart. Fill the email field. Try to complete the purchase with one hand. This is the one that gets skipped, and the only one that tests the thing you are actually selling.
Where the speed conversation and the usability one get merged
A note on something that caused me to look at this at all.
A post I read listed six PDP speed problems — render-blocking scripts, an unpreloaded LCP image, oversized PNGs, unlazy-loaded video, apps fetching critical product data, off-screen content loading early. Specific, from someone who does this full-time.
Five of the six are properties of the page. One is not: an app that fetches critical product data owns part of your critical path, and you cannot fix it by editing the page. The author’s own aside admits the same about render-blocking scripts — “nearly impossible with most Shopify apps.”
So two of six resolve to the same owner, and it is not the theme.
Two diagnostics from that post are worth repeating because a non-technical person can run them without a tool.
Run this yourself
Two readings you can take on your own product page
- Load the page and watch the main product image Page loads, main image appears seconds later → the LCP image is not prioritised.
- Load the page and watch the variant dropdowns Page loads, dropdowns appear seconds later → something is doing a round-trip to an external source before the page is usable.
- Search Console → Core Web Vitals → the mobile LCP distribution A passing lab score with a failing field distribution means you have been optimising the wrong instrument.
- Open your own store from a link inside Instagram, on your phone If navigation past the first page breaks, the in-app browser is a surface nobody has tested.
Nearly the same observation, different causes, different fixes. Which element is late is the test.
What I have not worked out
Being exact about this, because the whole point of separating four tests is to stop people claiming more than their test supports.
- I have not measured a store before and after a mobile fix. No conversion number in this piece, because I do not have one.
- I do not know the hit rate. How often a store passes in the lab and fails in the field is a countable thing, and I have not counted it.
- The in-app browser evidence is reviews, not measurement — four of them, one theme.
- I have not profiled a PDP’s critical path, so the app-ownership point is a reading of someone else’s list.
What is solid: the 2.5s threshold is a 75th-percentile field metric and a lab run cannot evaluate it, and a resized desktop window changes width and nothing else. Those are checkable in the specifications, not claims about your store.
If your current answer to “is the store fast?” came from one test on your own machine, the useful next step is not a fix. It is finding out which of the four questions you answered.
Sources
- Largest Contentful Paint (LCP) — the 75th-percentile threshold web.dev (Google)