Design a safe file-type detector

Design the component that decides what an uploaded file really is: magic-byte signatures at fixed offsets, refusal of ambiguous and polyglot files, markup sniffing so browsers cannot reinterpret an image as HTML, structural end checks, and a verdict the rest of the upload pipeline can trust.

● Medium🔒 PROBy ·Design problem·17 min read·Intermediate → Advanced
file uploadmagic bytescontent sniffingpolyglot filesnosniffMIME type

01Understand the problem

✓What you will learn
  • Identify file types from content using signatures at fixed offsets
  • Detect polyglots and files that a browser would sniff as HTML
  • Return one unambiguous verdict or refuse
  • Serve the result so browsers cannot reinterpret it (nosniff, Content-Disposition, separate origin)
◎Interview radar

What they probe: why the extension and the client's Content-Type are worthless as evidence, what a magic number is and where it sits, what a polyglot is (GIFAR, PDF plus zip, JPEG with HTML in a comment), what browsers do with a file served without a type, and why SVG is not an image for security purposes. Where candidates slip: trusting file.type from the browser, checking signatures anywhere in the file, accepting SVG as an image, and serving uploads from the main origin.

The name and Content-Type of an upload are written by the client, so they are claims, not facts. The bytes are the only evidence. Most binary formats begin with a magic number: PNG with 89 50 4E 47 0D 0A 1A 0A, PDF with %PDF-, zip with PK\x03\x04. A detector reads the first few kilobytes and matches signatures at fixed offsets. The hard part is what to do when a file matches two things, or looks like a valid image and also like HTML.

  1. detect(head, tail) returns exactly one kind, or unknown or ambiguous.
  2. Signatures are anchored at their specified offset, never searched anywhere in the file.
  3. Any file whose first kilobyte contains HTML-like markup is flagged, whatever its signature.
  4. Formats with a defined end marker (PNG IEND) are checked for trailing data.
  5. The policy layer, not the detector, decides which kinds are allowed for a feature.
KindSignatureOffset
PNG89 50 4E 47 0D 0A 1A 0A0
JPEGFF D8 FF0
GIFGIF87a or GIF89a0
WebPRIFF then WEBP0 and 8
PDF%PDF-0 (readers tolerate it later in the first 1024 bytes; we do not)
ZIP (also DOCX, XLSX, JAR, APK)50 4B 03 040
MP4 / ISO BMFFftyp4

This is an advanced module

Problems: parsing and sandboxing components is part of the Pro pass. 4 more sections — the deep dives, the final design, follow-ups and the test — unlock with it.

Already a cyart.io or studio.cyart.io member? Your pass code is in your Studio account.