An infinite scroll collection page sends only its first batch of products in the raw HTML. Everything that loads afterward, as a shopper scrolls, runs on a script that fires on a user action. GPTBot and ClaudeBot don't scroll, and don't run that script, so a 500-product category page can exist to AI crawlers as 24 products.
What a Crawler Actually Receives
A browser scrolling your category page triggers a script that fetches more products and inserts them into the page. That script runs after a person acts. Google explains why this matters in its own pagination guidance. According to Google, "Google's crawlers don't 'click' buttons and generally don't trigger JavaScript functions that require user actions to update the current page contents." Googlebot renders JavaScript. It still misses scroll-triggered content for this reason.
AI crawlers go a step further. According to Vercel's December 2024 crawler analysis, GPTBot made 569 million requests across Vercel's network in a single month with zero JavaScript execution. ClaudeBot and PerplexityBot behave the same way. Gemini runs on Googlebot's infrastructure. Google states plainly that "Google Search does not interact with your page," so scroll-loaded products usually stay hidden from it too. Products below your first batch lose their main discovery path. If they aren't in your sitemap or linked from anywhere else on your site, a crawler has no route to them at all.
The Test: Fetch Raw HTML and Count Products
Fetch your category page's raw server HTML directly, the way a crawler receives it, and count the products in that response. Then compare that count against what a person sees after scrolling to the bottom in a browser. Check two more things while you're there: whether the raw HTML links to a page 2, and whether the hidden products are in your sitemap. The gap between what's in the response and what's on the page, minus whatever your sitemap still covers, is your real exposure.
This is the same raw-HTML test that catches accordion and tab content hidden from AI crawlers. Infinite scroll is a collection-page version of the same bug: content that renders for a person and never reaches the raw response a crawler reads.
Why This Costs You the Recommendation
Shopping answers come from product data, and product data starts with a crawler having some route to the product page. Say your best match for a query sits in row 40 of a category. A crawler only sees the first 24 rows, and there's no sitemap entry either. That product is invisible to the exact question it was built to answer, no matter how well the PDP is written.
Catalog size raises the stakes here. A 24-product batch limit costs a boutique brand little. It can cut the majority of a 2,000-SKU catalog out of every category page an AI crawler reads. That's the exact moment a larger catalog has the most to gain from AI-driven discovery.
The Fix: Paginated Loading with Real URLs
Google's own guidance recommends giving each chunk of results a unique, persistent URL, for example with a `?page=2` query parameter. That beats loading everything under one URL that never changes. Each of those URLs then carries its own products in its own raw HTML, readable by a crawler that never scrolls or clicks anything. The shopper still gets the infinite-scroll experience in the browser. The crawler gets a paginated archive underneath it.
What's Local and What Needs a Ticket
If you control your storefront's collection template directly, your own team can make the fix: add real, crawlable pagination URLs behind the existing scroll behavior. If your collection pages run on a shared platform template or a headless frontend your team doesn't own outright, the fix usually needs a ticket. It changes how the page loads its data, not just what the page displays.
FAQs
Does infinite scroll hurt Google indexing too?
It can. Googlebot renders JavaScript but doesn't scroll, so products that load only on a scroll event usually stay hidden from it too. The gap is smaller than with GPTBot, ClaudeBot and PerplexityBot, which don't render JavaScript at all, but it isn't zero.
How many products typically load in the first batch?
It depends on your platform and theme. Run the raw HTML test on your own category pages rather than assuming a number.
Does lazy-loading images cause the same problem?
Native lazy-loading (`loading="lazy"`) keeps the image tag in the raw HTML, so it's usually safe. Script-based image loaders that swap in the real image source later can hide images the same way infinite scroll hides products. Test each pattern on your site separately.
Is this only a problem for ecommerce catalogs?
It shows up anywhere a page loads a long list on scroll: blog archives, search results, review lists. It matters most for ecommerce, where a hidden product is a hidden sale.
The Bottom Line
Infinite scroll works by loading more content only after a person scrolls, and AI crawlers don't scroll. Fetch your own raw HTML and count what's actually there. Then route around the gap with paginated URLs your storefront can serve underneath the scroll experience shoppers already expect.
Want to know whether AI crawlers can see your whole catalog? We audit your site as part of our free AI visibility assessment. Then we tell you what the crawlers can and can't read.