Reading Google Play and App Store reviews straight from their JSON, no browser

I spent an afternoon last year fighting a headless Chrome that scraped Play Store reviews. CAPTCHA, then a proxy bill, then a week later the DOM shifted and the whole thing returned empty arrays. Threw it out. Turns out I never needed the browser. Both Google Play and the Apple App Store serve reviews as plain JSON you can hit with an HTTP request. No login, no proxies, no Playwright. This is the request shapes, the pagination caps that aren't in any docs, and the one place Google's format bit me. All Node.js, all on got-scraping — a drop-in got replacement that copies a real browser's TLS and header fingerprint. That fingerprint earns its keep. The identical request from stock axios or fetch would sometimes come back 403 while got-scraping walked right through, because Play is fingerprinting the TLS handshake, not reading your User-Agent. Apple first, because Apple made it easy Apple publishes reviews as an RSS feed in JSON. One endpoint: https://itunes.apple.com/{country}/rss/customerreviews/page={1-10}/id={appId}/sortby={mostrecent|mosthelpful}/json Enter fullscreen mode Exit fullscreen mode country — a storefront code (us, gb, de...). Every storefront keeps its own reviews. appId — the numeric id from the store URL: apps.apple.com/us/app/whatsapp-messenger/id310633997. page — 1 to 10, and 10 is the wall. Fifty reviews a page, so ~500 per storefront. Coming back to that. import { gotScraping } from 'got-scraping'; async function fetchAppleReviews(appId, { country = 'us', maxReviews = 200 } = {}) { const out = []; const pages = Math.min(10, Math.ceil(maxReviews / 50)); for (let page = 1; page ({ id: r[0], author: r[1][0], rating: r[2], text: r[4], date: new Date(r[5][0] * 1000).toISOString(), thumbsUp: r[6], reply: r[7]?.[1] ?? null, // developer reply text appVersion: r[10], })); const nextToken = data[1]?.[1] ?? null; return { reviews, nextToken }; } Enter fullscreen mode Exit fullscreen mode Then loop, feeding nextToken back until you've got enough or it comes back null: async function fetchPlayReviews(pkg, { maxReviews = 200 } = {}) { const out = []; let token = null; while (out.length < maxReviews) { const res = await gotScraping({ url: 'https://play.google.com/_/PlayStoreUi/data/batchexecute?hl=en&gl=us', method: 'POST', body: buildBody(pkg, { count: 150, token }), headers: { 'content-type': 'application/x-www-form-urlencoded;charset=UTF-8' }, }); const { reviews, nextToken } = parse(res.body); if (!reviews.length) break; out.push(...reviews); if (!nextToken) break; token = nextToken; } return out.slice(0, maxReviews); } Enter fullscreen mode Exit fullscreen mode A few things I only learned by running this against real apps: Play reviews carry no title. Just a rating and a body. Apple gives you both. If you're merging the two stores into one schema, title has to be nullable or you'll drop half your Play data on a strict validator. Developer replies hide in slot [7] on Play — text at [7][1], timestamp at [7][2][0]. Apple's public feed doesn't surface replies at all. The loop fires requests back to back with no delay. Proxyless, from my laptop and from a datacenter, I haven't been rate-limited doing this — but it's the assumption most likely to break at tens of thousands of reviews, and I'd put a throttle in front of it before trusting it at that scale. And here's where it got me. I first pulled appVersion from slot [8], eyeballed one response, saw a version string, shipped it. Some apps came back with a country code there instead. The version is [10]. The index parsing is brittle by design — Google can reshuffle slots whenever they like, and nothing tells you. So I pinned a test against a known app with a review I can eyeball and assert the fields on it. When Google moves something, that test screams before my users notice. Is this actually worth skipping the browser? For me, yes — and the number that convinced me: 250 Play reviews land in about 0.6 seconds this way. You're reading the exact API the store's own frontend reads, so a store redesign doesn't touch you. My Playwright version was 10 to 20 times slower, wanted a proxy budget the moment I scaled it, and died on the next UI refresh. I don't miss it. If you'd rather not babysit the slot indices I bundled both stores into one actor on Apify — one schema, handles the pagination and the batchexecute envelope, runs proxyless: App Reviews Scraper. It's mostly there so I stop re-fixing the [10]-versus-[8] kind of thing every quarter. But the code above is the whole trick, and rolling your own is very doable. The batchexecute envelope eats afternoons if you go in blind — if you get stuck on it, leave a comment and I'll dig in.