Design an SSRF-safe URL fetcher

Design the component behind webhooks, link previews and import-from-URL: a scheme and port allow-list, DNS resolution followed by an IP check that covers private, link-local, cloud metadata, IPv6 and IPv4-mapped forms, a check that runs inside the connection's own DNS lookup so rebinding cannot slip between check and connect, redirects re-checked on every hop, and hard size and time limits.

● Must know for interviews● HardFREEDesign problem·22 min read·Intermediate → Advanced

SSRFDNS rebinding169.254.169.254IMDSIPv4-mapped IPv6webhook security

01Understand the problem

✓What you will learn

Explain server-side request forgery and why the cloud metadata endpoint makes it critical
Check the resolved IP, not the hostname, against a block list that covers every special range
Defeat DNS rebinding by checking the address the socket actually connects to
Re-validate redirects and bound size and time

◎Interview radar

What they probe: whether you check before or after DNS, what happens between your check and the connect, whether http://[::ffff:169.254.169.254]/ or http://2852039166/ gets through, how redirects are handled, and what a defence in depth looks like (an egress proxy, IMDSv2). Where candidates slip: a regex on the hostname, resolving once and letting the HTTP client resolve again, following redirects with the default client, forgetting IPv6.

SSRF (server-side request forgery) is when an attacker makes your server send a request on their behalf. Your server sits inside the network, so it can reach things the attacker cannot: admin panels on 10.x addresses, Redis on localhost and, above all, the cloud instance metadata service at 169.254.169.254, which hands out temporary credentials for the machine's role. The 2019 Capital One breach was SSRF to that endpoint.

fetch(url) returns status, headers, body and the final URL, or a typed error.
Schemes: https: (and http: only if configured). Ports: 443 and 80 unless configured. No user:pass@ in URLs.
Every address the host resolves to must be public; one bad address fails the whole request.
The address checked is the address connected to, on every hop, including redirects (at most 3).
Body capped at 5 MB, whole operation at 10 seconds, no decompression.

Bypass attempt	What the URL parser or DNS turns it into	Caught by
`http://169.254.169.254/latest/meta-data/`	Link-local IPv4	169.254.0.0/16 in the block list
`http://2852039166/`	169.254.169.254 (decimal integer form)	WHATWG URL normalises it before the check
`http://0x7f.1/`	127.0.0.1	Normalised, then 127.0.0.0/8
`http://[::ffff:169.254.169.254]/`	`[::ffff:a9fe:a9fe]`, IPv4-mapped IPv6	::ffff:0:0/96 blocked outright
`http://evil.example/` resolving to 10.0.0.5	Private address via DNS	Check inside the lookup
Rebinding: first answer public, second 127.0.0.1, TTL 0	Two different resolutions	Only one resolution exists: the one the socket uses
Public URL that redirects to `http://localhost:6379/`	Redirect target	Every hop goes through the same checks

02Interfaces first

The contractts

import type { IncomingHttpHeaders } from "node:http";

export type FetchErrorCode = "bad_url" | "scheme" | "credentials" | "port" | "blocked_address" | "dns"
  | "too_large" | "timeout" | "too_many_redirects" | "network";

export class FetchError extends Error {
  constructor(readonly code: FetchErrorCode, message: string) { super(message); this.name = "FetchError"; }
}

export interface FetchPolicy {
  schemes: readonly ("https:" | "http:")[];
  ports: readonly number[];
  maxBytes: number;          // 5_000_000
  timeoutMs: number;         // whole operation, every redirect included
  maxRedirects: number;      // 3
  userAgent: string;
}

export interface FetchResult { status: number; headers: IncomingHttpHeaders; body: Buffer; finalUrl: string }

/** Injectable so tests can simulate DNS answers, including rebinding. */
export type Resolve = (host: string, family: 0 | 4 | 6) => Promise<{ address: string; family: number }[]>;

iWhy one bad address fails the request

If a name resolves to one public and one private address, the client may connect to either, and Happy Eyeballs may race both. Filtering out the bad one is tempting but lets an attacker steer you with record ordering. Refusing the whole name is simpler to reason about.

03Block list, guarded lookup and the fetcher

FigWhere the check livesSchematic

ACTOR
EDGE
SERVICE
DATA
Live flow
Lookup / async

//The URL is parsed and checked, then the HTTP client opens a socket. The client's DNS lookup is replaced with a guarded one that resolves, checks every address and returns only checked addresses, so the address that was checked is the address connected to. Redirects loop back to the start.

Block list and guarded lookupts

import { BlockList, isIP, type LookupFunction } from "node:net";
import { lookup as dnsLookup } from "node:dns/promises";
import http from "node:http";
import https from "node:https";

const V4: [string, number][] = [
  ["0.0.0.0", 8], ["10.0.0.0", 8], ["100.64.0.0", 10], ["127.0.0.0", 8], ["169.254.0.0", 16], ["172.16.0.0", 12],
  ["192.0.0.0", 24], ["192.0.2.0", 24], ["192.88.99.0", 24], ["192.168.0.0", 16], ["198.18.0.0", 15],
  ["198.51.100.0", 24], ["203.0.113.0", 24], ["224.0.0.0", 4], ["240.0.0.0", 4],
];
const V6: [string, number][] = [
  ["::", 96],            // unspecified, loopback and deprecated IPv4-compatible (::127.0.0.1)
  ["::ffff:0:0", 96],    // IPv4-mapped: refuse outright rather than unwrap
  ["64:ff9b::", 96], ["64:ff9b:1::", 48],   // NAT64 can reach any IPv4 address, private ones included
  ["100::", 64], ["2001::", 32], ["2001:db8::", 32], ["2002::", 16],   // discard, Teredo, documentation, 6to4
  ["fc00::", 7],         // unique local, includes AWS IMDS over IPv6 (fd00:ec2::254)
  ["fe80::", 10], ["fec0::", 10], ["ff00::", 8],
];

// Two lists, never one: BlockList also matches IPv4 addresses against IPv6 rules through their mapped
// form, so ::ffff:0:0/96 in a shared list would block every IPv4 address on the internet.
const blockedV4 = new BlockList();
const blockedV6 = new BlockList();
for (const [a, n] of V4) blockedV4.addSubnet(a, n, "ipv4");
for (const [a, n] of V6) blockedV6.addSubnet(a, n, "ipv6");

export function isBlockedIp(ip: string): boolean {
  const fam = isIP(ip);
  if (fam === 0) return true;                                  // not an IP literal: never treat as safe
  return fam === 4 ? blockedV4.check(ip, "ipv4") : blockedV6.check(ip, "ipv6");
}

export function parseUrl(s: string, base?: URL): URL {
  if (s.length > 2048) throw new FetchError("bad_url", "URL too long");
  try { return new URL(s, base); } catch { throw new FetchError("bad_url", "unparseable URL"); }
}

/** Runs on the parsed URL of every hop. The WHATWG parser has already turned 2852039166 and 0x7f.1 into dotted quads. */
export function checkUrl(u: URL, policy: FetchPolicy): void {
  if (!policy.schemes.some((s) => s === u.protocol)) throw new FetchError("scheme", "scheme not allowed: " + u.protocol);
  if (u.username !== "" || u.password !== "") throw new FetchError("credentials", "userinfo in URL is not allowed");
  const port = u.port === "" ? (u.protocol === "https:" ? 443 : 80) : Number(u.port);
  if (!policy.ports.includes(port)) throw new FetchError("port", "port not allowed: " + port);
  const host = u.hostname.startsWith("[") ? u.hostname.slice(1, -1) : u.hostname;
  if (isIP(host) !== 0 && isBlockedIp(host)) throw new FetchError("blocked_address", "address not allowed: " + host);
}

/** Replaces the socket's DNS lookup. Node skips lookup for IP literals, which is why checkUrl covers those. */
export function guardedLookup(resolve: Resolve): LookupFunction {
  return (hostname, options, callback) => {
    const f = options.family;
    const fam = f === 4 || f === "IPv4" ? 4 : f === 6 || f === "IPv6" ? 6 : 0;
    resolve(hostname, fam).then(
      (addrs) => {
        if (addrs.length === 0) return callback(new FetchError("dns", "no addresses for " + hostname), "", 0);
        const bad = addrs.find((a) => isBlockedIp(a.address));
        if (bad) return callback(new FetchError("blocked_address", hostname + " resolves to " + bad.address), "", 0);
        if (options.all) return callback(null, addrs);           // Node 20+ asks for all addresses (autoSelectFamily)
        callback(null, addrs[0].address, addrs[0].family);
      },
      () => callback(new FetchError("dns", "cannot resolve " + hostname), "", 0),
    );
  };
}

The fetcherts

export class SafeFetcher {
  private readonly lookup: LookupFunction;
  constructor(private readonly policy: FetchPolicy, resolve: Resolve = (h, f) => dnsLookup(h, { all: true, family: f })) {
    if (!(policy.maxBytes > 0 && policy.timeoutMs > 0 && Number.isInteger(policy.maxRedirects) && policy.maxRedirects >= 0)) {
      throw new Error("invalid fetch policy");
    }
    this.lookup = guardedLookup(resolve);
  }

  async fetch(input: string): Promise<FetchResult> {
    const signal = AbortSignal.timeout(this.policy.timeoutMs);      // one budget for the whole chain
    let url = parseUrl(input);
    for (let hop = 0; hop <= this.policy.maxRedirects; hop++) {
      checkUrl(url, this.policy);
      const res = await this.once(url, signal);
      const location = res.headers.location;
      if (res.status >= 300 && res.status < 400 && location !== undefined) {
        url = parseUrl(location, url);                             // relative redirects resolve against the current hop
        continue;
      }
      return { ...res, finalUrl: url.href };
    }
    throw new FetchError("too_many_redirects", "more than " + this.policy.maxRedirects + " redirects");
  }

  private once(url: URL, signal: AbortSignal): Promise<Omit<FetchResult, "finalUrl">> {
    const mod = url.protocol === "https:" ? https : http;
    return new Promise((resolve, reject) => {
      const fail = (e: unknown) => reject(e instanceof FetchError ? e
        : signal.aborted ? new FetchError("timeout", "timed out") : new FetchError("network", String(e)));
      const req = mod.request(url, {
        method: "GET", lookup: this.lookup, signal, agent: false,
        headers: { "user-agent": this.policy.userAgent, accept: "*/*", "accept-encoding": "identity" },
      }, (res) => {
        const status = res.statusCode ?? 0;
        if (status >= 300 && status < 400) {                       // never read a redirect body
          res.destroy();
          return resolve({ status, headers: res.headers, body: Buffer.alloc(0) });
        }
        if (Number(res.headers["content-length"]) > this.policy.maxBytes) {
          res.destroy();
          return reject(new FetchError("too_large", "declared length over limit"));
        }
        const chunks: Buffer[] = [];
        let size = 0;
        res.on("data", (c: Buffer) => {
          size += c.length;
          if (size > this.policy.maxBytes) { res.destroy(); reject(new FetchError("too_large", "body over limit")); return; }
          chunks.push(c);
        });
        res.on("end", () => resolve({ status, headers: res.headers, body: Buffer.concat(chunks) }));
        res.on("error", fail);
        res.on("close", () => { if (!res.complete) fail(new Error("connection closed early")); });
      });
      req.on("error", fail);
      req.end();
    });
  }
}

▭On the whiteboard · Design choices4 points

check point

inside the socket's lookup, so check and connect use one resolution; IP literals checked on the parsed URL

redirects

manual, at most 3, each hop re-parsed and re-checked; redirect bodies never read

limits

one AbortSignal.timeout for the chain, Content-Length pre-check plus a streaming byte count, identity encoding only

TLS

the URL keeps the hostname, so SNI and certificate checks use the name while the socket uses the checked IP

04Rebinding, defence in depth and tests

DNS rebinding is the classic bypass of a naive check. The attacker's name server answers the first query with a public IP (your check passes) and the second with 127.0.0.1 and a TTL of zero (your HTTP client connects there). Resolving yourself and then handing the hostname to a client that resolves again is exactly this bug. Pinning by doing the check inside the client's own lookup leaves only one resolution to attack.

Layer	What it adds	Example
Application check (this lesson)	Precise errors, per-feature policy	SafeFetcher
Egress proxy	One enforcement point for every service, including code that forgot	Stripe's Smokescreen, or Squid with ACLs
Network	Fetch workers in a subnet that cannot route to internal ranges	Security groups, no route to 10.0.0.0/8
Metadata hardening	Stolen GETs no longer yield credentials	AWS IMDSv2 (PUT for a session token, hop limit 1), GCP requires Metadata-Flavor: Google

!Watch what the response is used for

Blind SSRF still lets an attacker port-scan by timing, and reflecting the fetched body back (a link preview showing raw HTML) turns SSRF into data theft. Return only what the feature needs: a title, a thumbnail you re-encode, a status code.

vitestts

import { describe, it, expect } from "vitest";

const policy: FetchPolicy = { schemes: ["https:", "http:"], ports: [80, 443], maxBytes: 5_000_000, timeoutMs: 5_000, maxRedirects: 3, userAgent: "cyart-fetch/1" };
const errCode = async (p: Promise<unknown>) => p.then(() => "ok", (e: FetchError) => e.code);

describe("SSRF guards", () => {
  it.each([
    "http://169.254.169.254/latest/meta-data/",
    "http://2852039166/",
    "http://0x7f.1/",
    "http://0/",
    "http://[::1]/",
    "http://[::ffff:169.254.169.254]/",
    "http://[fd00:ec2::254]/",
  ])("blocks %s before any socket opens", async (u) => {
    expect(await errCode(new SafeFetcher(policy).fetch(u))).toBe("blocked_address");
  });
  it("still allows ordinary public addresses", () => {
    expect(isBlockedIp("93.184.215.14")).toBe(false);
    expect(isBlockedIp("2606:4700::6810:84e5")).toBe(false);
  });
  it("refuses other schemes, ports and userinfo", async () => {
    const f = new SafeFetcher(policy);
    expect(await errCode(f.fetch("file:///etc/passwd"))).toBe("scheme");
    expect(await errCode(f.fetch("http://example.com:6379/"))).toBe("port");
    expect(await errCode(f.fetch("https://user:pw@example.com/"))).toBe("credentials");
  });
  it("blocks a name whose DNS answer is private, inside the connection's own lookup", async () => {
    const f = new SafeFetcher(policy, async () => [{ address: "93.184.215.14", family: 4 }, { address: "10.0.0.5", family: 4 }]);
    expect(await errCode(f.fetch("http://mixed.example/"))).toBe("blocked_address");
  });
  it("survives rebinding: the only resolution is the one the socket uses", async () => {
    let n = 0;
    const flip = async () => [{ address: n++ === 0 ? "93.184.215.14" : "127.0.0.1", family: 4 }];
    const lookup = guardedLookup(flip);
    const first = await new Promise((r) => lookup("rebind.example", {}, (err) => r(err)));
    const second = await new Promise((r) => lookup("rebind.example", {}, (err) => r(err)));
    expect(first).toBeNull();
    expect((second as FetchError).code).toBe("blocked_address");
  });
});

05Extensions, follow-ups, how to say it

Allow-lists beat block lists when the destinations are known: a webhook feature can require the customer to verify a domain, and an import feature can allow only a few providers.
Per-tenant egress identity: send webhooks from dedicated IPs and sign them, so receivers can verify you and you can rate limit per tenant.
Response handling: parse in a sandboxed worker, re-encode images, never render fetched HTML.
Observability: log the resolved IP, the hop chain and the block reason; a burst of blocked_address is an attack in progress.

Q1Why not check the hostname with a regex for 'localhost' and '127.'?+

Names resolve to anything, and IP literals have many spellings (decimal, hex, octal, IPv6, mapped). Only the resolved address is meaningful, and it must be checked at connect time.

Q2Your check passes but the HTTP library resolves the name again. What is the attack?+

DNS rebinding: a zero-TTL record answers public for the check and internal for the connect. The fix is one resolution, used for both, which the guarded lookup provides.

Q3Why not just follow redirects with the client's default behaviour?+

A public URL can 302 to http://169.254.169.254/. Default redirect following skips your checks; every hop has to pass the same validation.

Q4What does IMDSv2 change?+

Credentials need a session token from a PUT with a custom header, and the token response has a hop limit of 1 by default, so a simple GET-based SSRF or a container behind an extra hop cannot get credentials.

“How to say it in the room

"I parse with the WHATWG URL parser, which normalises odd IP spellings, then allow-list scheme and port and refuse userinfo. The real check is on addresses: I replace the HTTP client's DNS lookup with one that resolves, refuses the request if any address is private, loopback, link-local, metadata, unique local, NAT64 or IPv4-mapped, and hands only checked addresses to the socket, so rebinding has no second resolution to attack. Redirects are manual and re-checked, the whole chain shares one timeout, and the body is capped while streaming. Behind it sits an egress proxy and IMDSv2, because one forgotten code path should not be the whole defence."

Test yourself

A fetcher resolves the hostname with dns.lookup, checks the IP is public, then calls fetch(url) with the original hostname. What is the flaw?

Finished this one?

Tick it and it shows as done on the roadmap and the course.