A Screaming Frog proxy is one proxy server that carries every crawl request, set under File > Settings > Proxy (on macOS, Screaming Frog SEO Spider > Settings > Proxy). Tick Use Proxy Server, enter the host and HTTP port from your ProxyHive order, restart, and supply the username and password when asked, or allowlist your own IP on the order and skip credentials. A static ISP IP in the country you audit lets you see a site the way its local visitors and search engines do.

Steps checked against Screaming Frog's proxy guide and configuration user guide on 2026-09-30.

Why crawl through a proxy at all

Most crawls do not need one. Three situations do:

  • Geo-redirecting sites. Many international sites redirect by IP location: a visitor in Germany lands on /de/, a visitor in the US on /us/. Crawl from your office and you audit one country's version only. An ISP IP chosen by country, region, city and carrier lets you crawl the German storefront from Germany, then the French one from France, and compare redirects, hreflang and canonicals per market. The Germany locations page and its siblings list what each market offers.
  • Your office IP is rate limited or blocked by a site you are allowed to audit, often after an earlier aggressive crawl.
  • Staging behind an allowlist. Some clients allow a fixed IP into staging. A static IP you rent is easier to hand them than your changing office address.

An ISP proxy suits this: the address is registered to a consumer carrier, it stays the same for the term you buy, and you can buy exactly one. Residential rotates and has no location selection today, which is the opposite of what a country audit needs.

Before you start

  1. Order an ISP IP in the country you want to audit (or a datacenter IP if the site does not care where you come from) and open the order in the dashboard at https://app.proxyhive.io.
  2. Copy the HOST, the HTTP port, USERNAME and PASSWORD. Each IP is its own endpoint.
  3. Decide on authentication. Username and password works anywhere; an IP allowlist on the order lets your own public IP connect with no credentials, which avoids the prompt entirely.

Use the HTTP port. HTTPS pages stay encrypted end to end inside the tunnel the SEO Spider opens.

Screaming Frog proxy settings, step by step

1. Open the proxy settings

  • Windows and Linux: File > Settings > Proxy.
  • macOS: Screaming Frog SEO Spider > Settings > Proxy.

Older versions kept system settings under Configuration > System > Proxy. If your menus look like that, the fields are the same.

2. Enter the proxy

  1. Tick Use Proxy Server.
  2. Enter the address (HOST from your order) and the port (the HTTP port).
  3. Click OK and Restart, or close and restart the SEO Spider yourself. Screaming Frog's guide is explicit that proxy changes only take effect after a restart.

3. Authenticate

Recent versions show a Use Proxy Credentials option with Username and Password fields in the same window; fill them from your order. If your version has no such fields, or you leave them empty, the SEO Spider asks for the credentials in a dialog when the crawl starts and the proxy challenges it. Screaming Frog's proxy guide does not describe either screen, so check your own window rather than trusting a screenshot.

With an IP allowlist on the order, leave the credentials empty. Password or IP allowlist covers when each is the better choice.

Verify the exit IP

  1. Tick Store HTML under Configuration > Spider > Extraction, so the SEO Spider keeps each response.
  2. Switch to Mode > List, paste https://api.ipify.org?format=json as the only URL and start.
  3. Select the URL and open the View Source tab in the lower window. The ip in the response should be the address on your order.

If the crawl fails with a connection error, test the same credentials outside the SEO Spider with the curl proxy guide: if curl works, the problem is in the settings window, not the proxy.

Crawl speed: protect the IP and the site

With a proxy, every request comes from one address, so speed matters twice: for the site you crawl and for the IP you rent. Screaming Frog crawls at 5 threads by default "to not overload servers", and its speed documentation warns that raising threads can slow or crash a server.

  1. Go to Configuration > Speed.
  2. Set Max URI/s to 1 or 2 to start. Screaming Frog recommends controlling speed with this rather than Max Threads, which you can leave at the default.
  3. Watch the Response Codes tab. A run of 429 or 403 responses means slow down; carrying on burns the IP's reputation with that site for as long as you hold it.

Crawling hard enough to degrade a site is disallowed under our allowed-use policy, proxy or not.

JavaScript rendering through the proxy

Configuration > Spider > Rendering offers Text Only or JavaScript. JavaScript mode loads each page in headless Chrome, which fetches scripts, styles, images and third-party tags too, all through the proxy. Expect several times the requests per page and a slower crawl. That is fine on an ISP or datacenter IP priced per IP; on per-GB residential traffic it adds up. Render only when the content you audit is built client side.

Limits worth knowing

  • One proxy, no rotation. Screaming Frog's guide says only one proxy server can be configured. To audit a second country, change the proxy and restart.
  • No exceptions. All HTTP and HTTPS traffic goes through the proxy, or none does. There is no bypass list.
  • Not a rank tracker. The SEO Spider audits sites; it does not collect search results. For SERP collection see scraping Google search results and our SERP tracking use case.

Troubleshooting

  • Every URL returns "Connection Refused" or times out. Wrong host or port, a SOCKS5 port entered where HTTP is expected, or the restart never happened.
  • 407 Proxy Authentication Required, or the credential prompt keeps returning. Wrong username or password, or the order is on IP allowlist and your current public IP is not on it.
  • You still land on your home country's version. Check the exit IP first. If it is right, the site may redirect on Accept-Language or a cookie rather than IP; set the language header under Configuration > HTTP Header.
  • Lots of 429s. Too fast for one IP. Lower Max URI/s.