Datacenter vs Residential vs Mobile Proxies: How to Choose, Test and Budget

Datacenter vs residential vs mobile proxies is really a question about one thing: which network a website believes your request came from. This guide explains how each proxy type (plus ISP, or static residential, proxies) gets its IP addresses, how sites detect them, which one to use for which target, what a successful page really costs, and how to test a pool before you pay for it.

It is vendor-neutral. The only product section is near the end, and it is labeled.

Datacenter vs residential vs mobile proxies: the short answer

If you only remember one rule: start with the cheapest type that works on your target, measure, and move up only for the domains that force you to.

How each proxy type works

Every public IP address belongs to a block that is announced on the internet by an autonomous system, identified by an ASN. Anyone can look up which organization owns that ASN and what kind of network it is. That lookup is the first thing a bot-detection system does, and it explains most of the price differences below.

Datacenter proxies

Datacenter proxies run on servers at hosting and cloud providers. The IP ranges are registered to those companies, so an ASN lookup on an exit IP returns a hosting provider's name rather than a consumer ISP. Here is what that looks like for a well-known address:

{
  "ip": "8.8.8.8",
  "org": "AS15169 Google LLC",
  "country": "US"
}

Datacenter IPs are fast, stable and cheap, and you can usually buy them shared (other customers use the same IPs) or dedicated (only you do). Two details matter for scraping:

• Subnets get banned together. Providers sell addresses from contiguous ranges. A site that sees abuse from a few addresses in a /24 can block all 256, including yours.
• IPv6 is cheap for a reason. A single home connection usually gets a whole /64 of IPv6 addresses, so many sites rate-limit or block per /64. A pool of "millions" of IPv6 addresses from a handful of /64 ranges behaves like a handful of IPs. Many sites also don't accept IPv6 at all.

Residential proxies

Residential proxy IPs are assigned by consumer internet providers to home connections. Your request travels to the provider's gateway and leaves the internet through a real household line, typically a device whose owner installed an app or SDK and agreed to share bandwidth. To a target, the visit looks like it comes from a home broadband customer.

Most residential networks work as a gateway: you connect to one hostname and port, and the provider picks the exit. Per-request rotation and sticky sessions are usually controlled with options in the proxy username. Because the exit is someone's device, a sticky IP lasts only as long as that device stays online, so design for the IP disappearing mid-session. Speed varies with the household connection, and you pay per GB.

ISP proxies (static residential)

ISP proxies, also sold as static residential proxies, use IP ranges registered to internet providers but hosted on servers in data centers. An ASN lookup says ISP, latency is datacenter-fast, and the IP is yours for the month. That makes them the natural choice for work that needs one stable identity: logged-in sessions, account management and multi-step flows.

The weakness is that nothing else about them looks like a home. Connection-type databases may still classify the range as business or hosting, the ranges tend to be contiguous, and a single "household" that sends thousands of requests a day with no other traffic stands out. Treat them as better than datacenter, not as equal to rotating residential.

Mobile proxies and carrier-grade NAT

Mobile proxies route traffic through 4G and 5G carrier networks, either from real phones and modems the provider operates or from peer devices. What makes them special is carrier-grade NAT (CGNAT): carriers place many subscribers behind a shared pool of public IPv4 addresses, using an internal range reserved for this purpose (100.64.0.0/10, defined in RFC 6598). At any moment, one public carrier IP can represent many real people.

That is why mobile IPs are hard to block: banning one can lock out many paying customers, so most sites rate-limit carrier traffic gently instead. The exit IP changes when the modem reconnects or the carrier reassigns it, which providers expose as rotation on a timer or on demand. The costs are higher latency, smaller pools per location, the highest price per GB, and the fact that carriers are moving traffic to IPv6, where the shared-address effect is weaker.

How websites detect proxies

Detection is a score built from many signals, not a single check. These are the ones that depend on your proxy choice.

IP and ASN reputation

IP-intelligence databases classify every address by network type. MaxMind, for example, publishes a connection type database (cable/DSL, cellular, corporate, satellite) and an anonymous IP database with flags for hosting providers, public proxies and residential proxies. Anti-bot vendors add what they see across thousands of customer sites: an IP or subnet that misbehaved somewhere else arrives with a worse score.

Consistency checks

• Location vs locale. An IP in Germany with Accept-Language: en-US and a browser time zone of America/New_York is unusual.
• Session continuity. The same session cookie arriving from five IPs in a minute is something real users almost never do.
• Network stack vs User-Agent. A proxy terminates the TCP connection, so the target sees the proxy's operating system in the TCP handshake. A request that claims to be Chrome on Windows over a Linux TCP stack is a mismatch some systems check.

Behavior

Request rate per IP, per subnet and per session; pages visited without any of the assets a browser would load; perfectly regular timing; deep pagination with no navigation in between. Better IPs don't hide any of this.

Everything that isn't the IP

The TLS handshake (JA3 and JA4 fingerprints), HTTP/2 settings, header order and headless browser markers identify your client, whichever IP it uses. A default Python HTTP client behind a mobile proxy still looks like a default Python HTTP client. The guide to web scraping challenges covers fingerprinting and how to fix it.

Rotating vs sticky proxy sessions

Rotating means a new exit IP per request. Sticky means the same exit IP for a period the provider lets you set, typically measured in minutes. The right choice depends on whether the site expects continuity.

• Rotate for independent public pages: product pages, listings, articles, search queries. Each request stands alone, and spreading load across IPs keeps per-IP rates low.
• Stick for anything with state: logins, carts, checkouts, multi-step forms, and pagination that depends on cookies or tokens set on page one.

The rule that prevents most session blocks: one identity is one exit IP, one cookie jar, one User-Agent and one locale. Rotate between identities, never within one.

import uuid
import requests

GATEWAY = "gateway.example.com:8000"
USER, PASSWORD = "USER", "PASS"

def new_identity(locale="en-US"):
    """One identity = one sticky exit IP + one cookie jar + one consistent set of headers."""
    session_id = uuid.uuid4().hex[:12]
    # Many providers pin the exit IP with a session token in the username.
    # The exact syntax differs per provider: check its docs.
    proxy = f"http://{USER}-session-{session_id}:{PASSWORD}@{GATEWAY}"
    s = requests.Session()
    s.proxies = {"http": proxy, "https": proxy}
    s.headers.update({
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                      "(KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36",
        "Accept-Language": f"{locale},{locale.split('-')[0]};q=0.9",
    })
    return s

# Rotate between identities, never within one.
shopper = new_identity()
# shopper.get("https://shop.example/cart") ... every step of the flow uses the same session

import { fetch, ProxyAgent } from 'undici';
import { randomUUID } from 'node:crypto';

const GATEWAY = 'gateway.example.com:8000';
const [USER, PASSWORD] = ['USER', 'PASS'];

// One identity = one sticky exit IP + one cookie jar + one consistent set of headers.
function newIdentity(locale = 'en-US') {
  const sessionId = randomUUID().slice(0, 12);
  // Session syntax differs per provider: check its docs.
  const dispatcher = new ProxyAgent(`http://${USER}-session-${sessionId}:${PASSWORD}@${GATEWAY}`);
  const headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36',
    'accept-language': `${locale},${locale.split('-')[0]};q=0.9`,
  };
  const cookies = new Map(); // fetch keeps no cookies, so the identity does
  return async (url, init = {}) => {
    const cookie = [...cookies].map(([k, v]) => `${k}=${v}`).join('; ');
    const res = await fetch(url, { ...init, dispatcher, headers: { ...headers, ...(cookie && { cookie }) } });
    for (const c of res.headers.getSetCookie()) {
      const [pair] = c.split(';');
      const i = pair.indexOf('=');
      cookies.set(pair.slice(0, i).trim(), pair.slice(i + 1));
    }
    return res;
  };
}

const shopper = newIdentity();
// await shopper('https://shop.example/cart'); every step of the flow uses the same identity

Which proxy type to use: a decision table by target

Use this as a starting point, then let your own test results overrule it.

1. Test the cheapest plausible tier with the script in the testing section, on 200 or more real URLs per target.
2. Fix the client before the network. If blocks show a fingerprint pattern (instant 403s on the first request from fresh IPs), a better IP won't help.
3. Escalate one domain at a time. Keep easy domains on the cheap tier; route only the hard ones to residential or mobile.
4. Re-test monthly. Sites change their protection, and pools degrade.

Proxy pricing: per GB vs per IP vs per request

Proxy providers bill in four ways, and the billing model changes which of your choices drive cost:

• Per IP per month: typical for datacenter and ISP proxies, often with generous or unmetered bandwidth. Cost grows with how many IPs you need, which grows with your request rate.
• Per GB: typical for residential and mobile. Cost grows with bytes transferred, including failed attempts, retries and every image a browser downloads.
• Per modem or port: for dedicated mobile proxies. You rent a device's connection for a period.
• Per successful request: how scraping APIs usually bill. Page weight stops mattering, and with some providers failures are free.

Per IP is usually the cheapest way to move bytes, residential per-GB costs more, and mobile costs the most. Actual rates vary widely with volume, geography and commitment, so get quotes for your own volume rather than trusting a table in a blog post.

A worked example (with illustrative numbers)

The number that matters is cost per successful page. Under per-GB billing it is:

cost per good page = (KB per attempt ÷ 1,000,000) × price per GB ÷ success rate

The script below compares three ways of fetching 100,000 product pages a day. Every number in it is an assumption chosen for easy arithmetic, not a market price or a measurement. The $4 per GB is a placeholder; replace it, the page weights and the success rates with your own quote and your own test results.

# Cost per successful page under per-GB and per-request pricing.
# Every input is an assumption: replace them with your own measurements and quotes.

def per_gb_cost(pages, kb_per_attempt, success_rate, price_per_gb):
    """Per-GB billing: you pay for every byte, including failed attempts."""
    attempts = pages / success_rate
    gb = attempts * kb_per_attempt / 1_000_000  # 1 GB = 1,000,000 KB here
    return gb, gb * price_per_gb

def per_request_cost(pages, price_per_request, success_rate, failures_billed):
    """Per-request billing: page weight is irrelevant; failures may or may not be billed."""
    attempts = pages / success_rate if failures_billed else pages
    return attempts * price_per_request

PAGES = 100_000          # successful pages per day
PRICE_PER_GB = 4.00      # ILLUSTRATIVE ONLY, not a quote

scenarios = [
    ("HTTP client, HTML only",          150,   0.85),
    ("Headless browser, all assets",    3_000, 0.92),
    ("Headless browser, assets blocked", 700,  0.92),
]
for name, kb, ok in scenarios:
    gb, usd = per_gb_cost(PAGES, kb, ok, PRICE_PER_GB)
    print(f"{name:34} {gb:8,.1f} GB/day  ${usd:9,.2f}/day  ${usd / PAGES * 1000:6.2f} per 1k pages")

# Break-even: the page weight at which per-GB and per-request cost the same.
price_per_request = 0.002  # ILLUSTRATIVE ONLY
kb = price_per_request * 0.92 * 1_000_000 / PRICE_PER_GB
print(f"\nbreak-even page weight at these prices: {kb:,.0f} KB per attempt")

HTTP client, HTML only                 17.6 GB/day  $    70.59/day  $  0.71 per 1k pages
Headless browser, all assets          326.1 GB/day  $ 1,304.35/day  $ 13.04 per 1k pages
Headless browser, assets blocked       76.1 GB/day  $   304.35/day  $  3.04 per 1k pages

break-even page weight at these prices: 460 KB per attempt

Three things fall out of the arithmetic, and they hold at any real price:

1. Rendering through a per-GB proxy multiplies the bill. A browser loads images, fonts, scripts and trackers, and every byte crosses the proxy. In this example the same pages cost about 18 times more to render than to fetch as HTML.
2. Blocking heavy resources is the biggest single saving. Cutting the rendered page from 3 MB to 700 KB cut the bill by more than three quarters. Measure your real page weight in the browser's network panel.
3. Success rate is a multiplier. At 50% success you pay for every good page twice, and retries make it worse.

Blocking heavy resources in Playwright takes a few lines:

# pip install playwright && playwright install chromium
from playwright.sync_api import sync_playwright

HEAVY = {"image", "media", "font"}

with sync_playwright() as p:
    browser = p.chromium.launch(proxy={"server": "http://gateway.example.com:8000",
                                       "username": "USER", "password": "PASS"})
    page = browser.new_page()
    page.route("**/*", lambda route: route.abort()
               if route.request.resource_type in HEAVY else route.continue_())
    page.goto("https://books.toscrape.com/", wait_until="domcontentloaded")
    print(page.title())
    browser.close()

// npm install playwright && npx playwright install chromium
import { chromium } from 'playwright';

const HEAVY = new Set(['image', 'media', 'font']);
const browser = await chromium.launch({
  proxy: { server: 'http://gateway.example.com:8000', username: 'USER', password: 'PASS' },
});
const page = await browser.newPage();
await page.route('**/*', (route) =>
  HEAVY.has(route.request().resourceType()) ? route.abort() : route.continue(),
);
await page.goto('https://books.toscrape.com/', { waitUntil: 'domcontentloaded' });
console.log(await page.title());
await browser.close();

How to test a proxy pool before you buy

Most providers offer a trial or a small starter package. Use it to measure the pool on your targets, under the conditions you will run in production. Advertised pool sizes and generic success rates tell you very little about your sites.

What to measure

Success rate

Status 200 and a marker that only appears on the real page

Block rate

By signature: challenge pages, CAPTCHAs, 403, 429

Errors

Proxy errors and timeouts, which are the provider's fault

Latency

p50, p90, p95 and p99 of successful requests

Exit IPs

Distinct IPs you actually get, and their ASNs

Bytes per good page

What you will pay for under per-GB billing

• Use real URLs from each target, not the provider's demo page or an IP-echo service.
• Send the same headers, client and rendering mode you will use in production.
• Run at production-like concurrency, and again at a different time of day.
• Collect at least 200 requests per target. With 50, a single request moves the success rate by two points.
• Run the same script on your own connection first as a baseline, then on each provider.

A proxy test script in Python

This script needs only requests. It checks exit-IP diversity, fetches each target repeatedly through the proxy, classifies every response, and prints success rate, outcomes and latency percentiles per target.

"""Measure a proxy pool against the sites you actually scrape.

pip install requests
PROXY_URL=http://user:[email protected]:8000 python proxy_test.py
"""
import os
import re
import statistics
import time
from collections import Counter
from concurrent.futures import ThreadPoolExecutor

import requests

PROXY_URL = os.environ.get("PROXY_URL")  # unset = test your own connection as a baseline
PROXIES = {"http": PROXY_URL, "https": PROXY_URL} if PROXY_URL else None

# Real pages from your targets, plus a string that only appears on a good page.
TARGETS = [
    ("https://books.toscrape.com/", "product_pod"),
    ("https://quotes.toscrape.com/", 'class="quote"'),
]
REQUESTS_PER_TARGET = 50
CONCURRENCY = 5
TIMEOUT = 30

HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
                  "(KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Language": "en-US,en;q=0.9",
}

# Heuristics, not guarantees. Add the signatures you see on your own targets.
BLOCK_SIGNATURES = {
    "cloudflare": re.compile(r"cf-chl|challenge-platform|Just a moment\.\.\.|Attention Required! \| Cloudflare", re.I),
    "datadome": re.compile(r"captcha-delivery\.com|datadome", re.I),
    "human/perimeterx": re.compile(r"px-captcha|_pxCaptcha|perimeterx", re.I),
    "akamai": re.compile(r"Access Denied.{0,600}Reference(?: |\s)(?:#|#)", re.I | re.S),
    "generic-captcha": re.compile(r"g-recaptcha|h-captcha|hcaptcha\.com|are you a robot|unusual traffic", re.I),
}


def classify(resp, marker):
    """Return 'ok' or a block label for one response."""
    if resp.headers.get("cf-mitigated") == "challenge":
        return "block:cloudflare"
    body = resp.text[:200_000]
    for name, pattern in BLOCK_SIGNATURES.items():
        if pattern.search(body):
            return f"block:{name}"
    if resp.status_code in (403, 429, 503):
        return f"block:http-{resp.status_code}"
    if resp.status_code != 200:
        return f"http-{resp.status_code}"
    if marker not in body:
        return "soft-fail:marker-missing"  # 200, but not the page you wanted
    return "ok"


def one(job):
    url, marker = job
    start = time.perf_counter()
    try:
        resp = requests.get(url, headers=HEADERS, proxies=PROXIES, timeout=TIMEOUT)
        outcome = classify(resp, marker)
        size = len(resp.content)
    except requests.exceptions.ProxyError:
        outcome, size = "error:proxy", 0
    except requests.exceptions.Timeout:
        outcome, size = "error:timeout", 0
    except requests.exceptions.RequestException as exc:
        outcome, size = f"error:{type(exc).__name__}", 0
    return url, outcome, time.perf_counter() - start, size


def exit_ips(n=20):
    """How many distinct exit IPs do n requests actually get?"""
    seen = Counter()
    for _ in range(n):
        try:
            ip = requests.get("https://api.ipify.org", proxies=PROXIES, timeout=TIMEOUT).text.strip()
            seen[ip] += 1
        except requests.RequestException:
            seen["(error)"] += 1
    return seen


def pct(values, p):
    if len(values) < 2:
        return values[0] if values else float("nan")
    return statistics.quantiles(values, n=100, method="inclusive")[p - 1]


if __name__ == "__main__":
    ips = exit_ips()
    print(f"exit IPs: {len(ips)} distinct in {sum(ips.values())} requests")

    jobs = [t for t in TARGETS for _ in range(REQUESTS_PER_TARGET)]
    with ThreadPoolExecutor(max_workers=CONCURRENCY) as pool:
        results = list(pool.map(one, jobs))

    for url, _ in TARGETS:
        rows = [r for r in results if r[0] == url]
        ok = [r for r in rows if r[1] == "ok"]
        lat = sorted(r[2] for r in ok)
        print(f"\n{url}")
        print(f"  success  {len(ok)}/{len(rows)} = {len(ok) / len(rows):.0%}")
        print(f"  outcomes {dict(Counter(r[1] for r in rows))}")
        if ok:
            print(f"  latency  p50 {pct(lat, 50):.2f}s  p90 {pct(lat, 90):.2f}s  "
                  f"p95 {pct(lat, 95):.2f}s  p99 {pct(lat, 99):.2f}s")
            print(f"  size     {statistics.mean(r[3] for r in ok) / 1024:.0f} KB per good page")

A short baseline run (8 requests per target, through a local test proxy, against two public scraping sandboxes) printed this. One exit IP is expected through a local proxy; a residential gateway should show close to one IP per request. Your real runs should be larger and should use your own targets.

exit IPs: 1 distinct in 20 requests

https://books.toscrape.com/
  success  8/8 = 100%
  outcomes {'ok': 8}
  latency  p50 0.22s  p90 0.24s  p95 0.24s  p99 0.24s
  size     50 KB per good page

https://quotes.toscrape.com/
  success  8/8 = 100%
  outcomes {'ok': 8}
  latency  p50 0.13s  p90 0.14s  p95 0.14s  p99 0.14s
  size     11 KB per good page

The same test in Node.js

Node's built-in fetch doesn't read proxy settings by itself, so this version uses ProxyAgent from undici.

// npm install undici
// PROXY_URL=http://user:[email protected]:8000 node proxy-test.mjs
import { fetch, ProxyAgent } from 'undici';

const dispatcher = process.env.PROXY_URL ? new ProxyAgent(process.env.PROXY_URL) : undefined;
const TARGETS = [
  ['https://books.toscrape.com/', 'product_pod'],
  ['https://quotes.toscrape.com/', 'class="quote"'],
];
const PER_TARGET = 50;
const CONCURRENCY = 5;
const HEADERS = {
  'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36',
  'accept-language': 'en-US,en;q=0.9',
};
const SIGNATURES = {
  cloudflare: /cf-chl|challenge-platform|Just a moment\.\.\./i,
  datadome: /captcha-delivery\.com|datadome/i,
  'human/perimeterx': /px-captcha|_pxCaptcha|perimeterx/i,
  akamai: /Access Denied[\s\S]{0,600}Reference(?: |\s)(?:#|#)/i,
  'generic-captcha': /g-recaptcha|h-captcha|hcaptcha\.com|are you a robot|unusual traffic/i,
};

function classify(res, body, marker) {
  if (res.headers.get('cf-mitigated') === 'challenge') return 'block:cloudflare';
  for (const [name, re] of Object.entries(SIGNATURES)) if (re.test(body)) return `block:${name}`;
  if ([403, 429, 503].includes(res.status)) return `block:http-${res.status}`;
  if (res.status !== 200) return `http-${res.status}`;
  return body.includes(marker) ? 'ok' : 'soft-fail:marker-missing';
}

async function one([url, marker]) {
  const start = performance.now();
  try {
    const res = await fetch(url, { headers: HEADERS, dispatcher, signal: AbortSignal.timeout(30_000) });
    const body = await res.text();
    return { url, outcome: classify(res, body, marker), ms: performance.now() - start, bytes: body.length };
  } catch (err) {
    return { url, outcome: `error:${err.name}`, ms: performance.now() - start, bytes: 0 };
  }
}

const pct = (sorted, p) => sorted[Math.min(sorted.length - 1, Math.ceil((p / 100) * sorted.length) - 1)];

const jobs = TARGETS.flatMap((t) => Array(PER_TARGET).fill(t));
const results = [];
await Promise.all(Array.from({ length: CONCURRENCY }, async () => {
  while (jobs.length) results.push(await one(jobs.shift()));
}));

for (const [url] of TARGETS) {
  const rows = results.filter((r) => r.url === url);
  const ok = rows.filter((r) => r.outcome === 'ok');
  const lat = ok.map((r) => r.ms / 1000).sort((a, b) => a - b);
  const outcomes = Object.groupBy(rows, (r) => r.outcome);
  console.log(`\n${url}\n  success  ${ok.length}/${rows.length} = ${Math.round((100 * ok.length) / rows.length)}%`);
  console.log('  outcomes', Object.fromEntries(Object.entries(outcomes).map(([k, v]) => [k, v.length])));
  if (lat.length) console.log(`  latency  p50 ${pct(lat, 50).toFixed(2)}s  p90 ${pct(lat, 90).toFixed(2)}s  p95 ${pct(lat, 95).toFixed(2)}s  p99 ${pct(lat, 99).toFixed(2)}s`);
}

Check what network the exit IPs belong to

A "residential" pool whose exits resolve to hosting companies is not residential. Sample a few dozen exit IPs and look them up:

import os
import requests

proxy = os.environ.get("PROXY_URL")
proxies = {"http": proxy, "https": proxy} if proxy else None

ip = requests.get("https://api.ipify.org", proxies=proxies, timeout=30).text.strip()
info = requests.get(f"https://ipinfo.io/{ip}/json", timeout=30).json()  # add ?token=... for volume
print(ip, info.get("org"), info.get("country"))

import { fetch, ProxyAgent } from 'undici';

const dispatcher = process.env.PROXY_URL ? new ProxyAgent(process.env.PROXY_URL) : undefined;
const ip = (await (await fetch('https://api.ipify.org', { dispatcher })).text()).trim();
const info = await (await fetch(`https://ipinfo.io/${ip}/json`)).json(); // add ?token=... for volume
console.log(ip, info.org, info.country);

How to read the results

Common proxy mistakes

1. Buying the most expensive tier first. Mobile proxies on a site that answers datacenter IPs happily is money with no benefit. Test upward.
2. Treating a 200 as success. Challenge pages, consent walls and empty shells often come back as 200. Check for a content marker.
3. Rotating inside a session. Changing IP between login and the next page, or between page 1 and page 2 of a cookie-based listing, is a strong bot signal.
4. Mismatched geography. A US IP with a German Accept-Language and a UTC time zone looks wrong. Keep IP, language and time zone consistent.
5. Rendering everything through per-GB bandwidth. Block images, fonts and media, and fetch JSON endpoints directly where the site has them.
6. Retrying blocked requests immediately on the same identity. It burns IPs and teaches the site your pattern. Back off, then retry on a different identity.
7. Expecting the proxy to fix fingerprinting. If the TLS handshake says Python, the IP doesn't matter much.
8. Buying on advertised pool size. What counts is how many distinct, clean exits you get on your targets, in your regions.
9. Leaking credentials. Proxy URLs contain passwords. Keep them in environment variables, not in code or logs.

Where a managed scraping API fits

This section is about ScrapingBot. A scraping API makes sense when proxies are a means to an end: you want the page, not a proxy operation. You pick how much protection a request needs, and the pools, rotation, retries and replacement happen behind the API.

On the ScrapingBot website scraping API, the proxy tier is one parameter:

Costs don't stack: the most capable option sets the price, so premium_proxy with render_js is still 10 credits. AI extraction adds 5. Blocked attempts (403, 429 and other block answers), timeouts and 5xx errors are refunded; a target 404 is a completed scrape and is charged. The credits and charging rules have the details.

Because pricing is per request, page weight doesn't matter: a 3 MB rendered page through residential IPs costs the same 10 credits as a 50 KB one. At published plan prices, assuming you use all of your monthly credits, that works out to:

A real premium_proxy response, trimmed:

{
  "success": true,
  "url": "https://books.toscrape.com/",
  "html": "<!DOCTYPE html>\n<!--[if lt IE 7]> …",
  "status": 200,
  "duration": "0.99",
  "credits_used": 10,
  "job_id": "9b6f994e-ecc6-4595-b04a-6c8a38ddff64"
}

Since failed rungs are refunded, the escalation path from the decision table becomes a loop: try the cheapest mode, check the content, step up only if the page didn't come back.

import os
import requests

API = "https://scrapingbot.io/api/v1/scrape"
API_KEY = os.environ.get("SCRAPINGBOT_API_KEY", "YOUR_API_KEY")

# Cheapest first. Blocked attempts are refunded, so a failed rung costs nothing.
TIERS = [
    {},                         # plain HTTP, 1 credit
    {"premium_proxy": True},    # residential IPs, 10 credits
    {"stealth_proxy": True},    # strongest option, always renders, 75 credits
]

def fetch(url, marker, extra=None):
    for tier in TIERS:
        body = {"url": url, **tier, **(extra or {})}
        r = requests.post(API, headers={"x-api-key": API_KEY}, json=body, timeout=90)
        data = r.json()  # read the body even on non-2xx: status mirrors the target
        if data.get("success") and marker in data.get("html", ""):
            return tier, data
        if data.get("fault") == "user":  # 400/404 from the target: escalating won't help
            return tier, data
    return None, data

tier, data = fetch("https://books.toscrape.com/", "product_pod")
print(tier or "plain", data["status"], data["credits_used"])

const API = 'https://scrapingbot.io/api/v1/scrape';
const API_KEY = process.env.SCRAPINGBOT_API_KEY ?? 'YOUR_API_KEY';

// Cheapest first. Blocked attempts are refunded, so a failed rung costs nothing.
const TIERS = [{}, { premium_proxy: true }, { stealth_proxy: true }];

async function fetchPage(url, marker, extra = {}) {
  let data;
  for (const tier of TIERS) {
    const res = await fetch(API, {
      method: 'POST',
      headers: { 'x-api-key': API_KEY, 'content-type': 'application/json' },
      body: JSON.stringify({ url, ...tier, ...extra }),
      signal: AbortSignal.timeout(90_000),
    });
    data = await res.json(); // read the body even on non-2xx: status mirrors the target
    if (data.success && data.html?.includes(marker)) return { tier, data };
    if (data.fault === 'user') return { tier, data }; // 400/404: escalating won't help
  }
  return { tier: null, data };
}

const { tier, data } = await fetchPage('https://books.toscrape.com/', 'product_pod');
console.log(tier, data.status, data.credits_used);

Both versions printed a plain-tier success for 1 credit on the sandbox page, which is the point: most pages never need the expensive rungs. Remember which tier each domain needed and start there next time.

For some targets the better answer is not a proxy tier at all but structured data: the Google Search API returns organic results as JSON, and the Amazon API, TikTok API and Instagram API do the same for those platforms. To turn a working curl command into Python or Node, paste it into the curl converter. For a wider comparison of hosted options, see the best web scraping APIs in 2026.

FAQ

What is the difference between datacenter and residential proxies?

Datacenter proxy IPs belong to hosting and cloud companies, so anyone who looks up the network (the ASN) sees a server, not a household. Residential proxy IPs are assigned by consumer internet providers to home connections, and the traffic leaves through a real household line. Datacenter IPs are faster and much cheaper; residential IPs are trusted more by sites that score traffic by network type.

Are ISP proxies the same as static residential proxies?

Yes, the two names describe the same product. The IP addresses are registered to an internet provider, so an ASN lookup says ISP, but the machines sit in a data center. You get datacenter speed and a fixed IP you can keep for weeks, which suits logins and long sessions. They are easier to spot than true residential IPs because nothing else about them looks like a home connection.

Why are mobile proxies so hard to block?

Mobile carriers use carrier-grade NAT, which puts many subscribers behind a shared pool of public IP addresses. Blocking one carrier IP can block many real customers at once, so most sites rate-limit carrier traffic gently instead of banning it. That collateral-damage problem, not any special technology, is what makes mobile IPs effective and expensive.

Should I use rotating or sticky proxies for web scraping?

Rotate per request for independent public pages such as product pages, listings and search queries. Use a sticky session for anything that carries state: logins, carts, multi-step forms and pagination that relies on cookies. A site that sees the same cookie arrive from several IPs in a minute has an easy reason to block you.

How many proxies do I need for web scraping?

Divide the request rate you need by the rate one IP can sustain on that site without blocks. If you need 600 pages a minute and a target tolerates about 6 requests a minute per IP, you need roughly 100 IPs in rotation, plus headroom for IPs that get flagged. Measure the per-IP tolerance with a small test; it differs a lot between sites.

Why am I still blocked when using residential proxies?

Because the IP is only one signal. Sites also check the TLS and HTTP/2 fingerprint of your client, headless browser markers, whether your language and time zone match the IP location, whether cookies jump between IPs, and how fast you request pages. Fix the client fingerprint and the session consistency before buying a more expensive proxy tier.

Are residential proxies legal to use?

Using a proxy is legal in most countries; what matters is how the network was sourced and what you do through it. Choose providers that can show the people whose connections they route have given informed consent, and apply the same rules to proxied scraping as to any other: public data, polite rates, no personal data you do not need. This is general information, not legal advice.