File Upload & Deserialization Security

Two features that look unrelated in a product backlog share one property: both accept opaque bytes from a stranger and hand them to something that interprets structure. An upload endpoint gives an attacker a file on your infrastructure and a URL that serves it back. A deserialization call gives them influence over which objects your process constructs. Both classes have the same shape — external data selects a code path — and both are dramatically cheaper to prevent than to detect.

This guide covers content-based type validation, storage that cannot execute, an isolated serving origin, archive and document handling, and the deserialization rules that keep untrusted payloads from becoming instructions. It is part of Vulnerability Patterns & Web Mitigation Strategies. The serving controls lean on secure HTTP header configuration, and stored files that are rendered rather than downloaded connect directly to cross-site scripting mitigation.

Threat Anatomy

An upload flaw is rarely a single mistake; it is a chain in which each link is individually defensible. The file is accepted because the declared type looked fine. It is stored under the caller’s chosen name because that seemed friendly. It lands in a directory the web server serves because that was the simplest deployment. It is returned with a content type derived from its extension, from the same origin as the application. At that point a file the attacker wrote executes as first-party script, and every session cookie and same-origin API is reachable from it.

Deserialization compresses the same chain into one call. A payload that can name a type gives the attacker a foothold in the object graph of the running process — object construction, property setters, and destructors all run before any application code inspects the result. The details differ by language; the invariant does not: if the format can express “construct this”, the parser is an execution primitive.

Four Links, Four Places to Break the Chain The chain runs from accepting the file on a declared type, through storing it under a caller-chosen name inside the served tree, to returning it with a guessed content type from the application origin, ending in script execution in first-party context. Each link is paired with the control that breaks it: content-based validation, generated names outside the web root, an explicit content type with a download disposition, and a separate serving origin. 1 · Accepted declared type looked plausible 2 · Stored caller's name, inside the served tree 3 · Served type guessed from the extension, same origin 4 · Executes first-party script, full session reach Sniff the bytes allowlist, then re-encode Generated name outside the web root Explicit type no sniffing, attachment Other origin no session reach Any one control breaks the chain — which is why an incident usually means all four were missing and why "we validate the extension" is never an adequate answer on its own

Prerequisites & Scope

  • A storage target that is not the application server’s filesystem — object storage, or at minimum a volume the web server has no route to.
  • A separate hostname available for user content, ideally on a domain that is not a subdomain of the application.
  • A content type allowlist per feature, derived from what the feature actually needs rather than what browsers can display.
  • An image or document processing library you are prepared to keep patched, because parsers are where the remaining risk lives.
  • A serialization audit: a list of every place the codebase parses a binary or object-graph format from an external source.

Out of scope: malware analysis workflows, digital rights management, and the storage-side encryption decisions that belong with your data classification work.

Mitigation Architecture

The target architecture separates three concerns that ad-hoc implementations conflate — accepting, storing and serving — and gives each its own control.

Concern Vulnerable pattern Hardened pattern
Accepting Trust the declared type and extension Detect type from content, allowlist, re-encode where possible
Naming Keep the caller’s filename Generate an opaque identifier; keep the original name as metadata only
Storing Write into the served directory tree Object storage or a non-served volume, with no execute permission
Serving Same origin, sniffed type Separate origin, explicit type, no-sniff, attachment disposition
Parsing Deserialize whatever arrives Value-only formats with a schema; no type names in the payload

The separation matters because each control fails differently. Type detection can be fooled by a polyglot file that is valid in two formats at once. Storage isolation cannot help if the serving path re-introduces the origin. And the serving origin does nothing for a file that is parsed server-side. Together they leave no single misjudgement that reaches execution.

Step-by-Step Implementation

Step 1 — Decide the type from the bytes (ASVS V12.1, V12.2) 

import magic                      # libmagic bindings
from PIL import Image

ALLOWED = {"image/png": ".png", "image/jpeg": ".jpg", "application/pdf": ".pdf"}

def accept_upload(stream, declared_name: str) -> tuple[bytes, str]:
    head = stream.read(8192)
    stream.seek(0)
    detected = magic.from_buffer(head, mime=True)      # from content, not the name
    if detected not in ALLOWED:
        raise Rejected(f"type {detected} is not accepted here")

    data = stream.read(MAX_BYTES + 1)
    if len(data) > MAX_BYTES:
        raise Rejected("file too large")

    if detected.startswith("image/"):
        data = reencode_image(data, detected)          # strips payloads that survive validation
    return data, ALLOWED[detected]

Re-encoding images is the highest-value step in this function. A file that is a valid PNG and carries an appended archive or a script comment stops being both once it has been decoded to a pixel buffer and written back out. It also strips metadata that frequently contains more personal information than the uploader realised.

Step 2 — Name it yourself and store it where nothing executes (ASVS V12.3) 

key = f"uploads/{tenant_id}/{uuid4().hex}{ext}"        # opaque, no caller input in the path
s3.put_object(
    Bucket=UPLOAD_BUCKET, Key=key, Body=data,
    ContentType=detected,                              # recorded, never re-derived later
    ContentDisposition=f'attachment; filename="{sanitised_display_name}"',
    Metadata={"original-name": sanitised_display_name, "uploaded-by": str(user_id)},
)

The caller’s filename never appears in a path. That single rule removes traversal, null-byte truncation, reserved device names, case-collision overwrites and unicode normalisation surprises in one move — the original name survives as a display attribute, which is all it was ever needed for.

Step 3 — Serve user content from an origin with nothing to steal (ASVS V12.5)

Deliver uploads from a hostname that shares no cookies, no storage and no session with the application, and state the type explicitly:

Content-Type: image/png
Content-Disposition: attachment; filename="invoice.png"
X-Content-Type-Options: nosniff
Content-Security-Policy: default-src 'none'; sandbox
Cross-Origin-Resource-Policy: same-site
Cache-Control: private, max-age=300

nosniff stops the browser from second-guessing the declared type; the policy and sandbox neutralise anything that does manage to be interpreted as a document; and the separate origin means that even in the worst case the executing content sits outside the application’s security context. Signed, short-lived URLs then re-attach authorization to a resource that is no longer protected by session cookies.

Step 4 — Refuse to deserialize untrusted object graphs (ASVS V5.5)

The rule is categorical: formats that can name a type must never be parsed from an untrusted source. That covers native object serialization in every major language, along with document formats configured to allow arbitrary tags.

# NEVER on untrusted input — the payload chooses which objects are built.
obj = pickle.loads(request.data)
obj = yaml.load(request.data)                # unsafe loader

# Value-only parsing, validated against a schema you control.
payload = json.loads(request.data)
model = UploadRequest.model_validate(payload)   # rejects unknown fields and wrong types
obj = yaml.safe_load(request.data)              # scalars, lists and maps only

Where a binary format is genuinely required — an internal message bus, a cache — bind it to a schema registry and sign the payload, so the parser is asked to build one known shape rather than whatever the sender named.

What the Format Lets the Sender Say Three tiers of data format. Value-only formats express scalars, lists and maps and are safe from any source once schema-validated. Schema-bound binary formats express one known shape and are acceptable from authenticated internal senders. Object-graph formats let the payload name types to construct and are acceptable only from a source you would trust to run code. Value-only — scalars, lists, maps safe from any source once validated against a schema you wrote; the payload cannot name anything Schema-bound binary — one known shape acceptable between authenticated internal services with a registry and a signature; not from the internet Object graph — the payload names types to construct acceptable only from a source you would let run code, because that is effectively what you are granting

Edge Cases & Bypass Patterns

Polyglot files

Four Upload Edge Cases and What Handles Each A polyglot file valid in two formats is defeated by re-encoding, because the output contains only what the decoder understood. Archive entries are constrained by path, count and size limits. Vector graphics are rasterised or served only from the isolated origin. Documents that fetch remote resources are converted with the network disabled. Polyglot — valid image and valid archive at once re-encode: the rewritten output contains only what the decoder parsed Archive entries that escape the extraction directory reject traversal, absolute paths, links and device files; cap count, ratio and total Vector graphics carrying script or remote references rasterise on upload, or serve exclusively from the isolated content origin Documents that fetch during conversion convert with the network disabled and a hard timeout — this is an outbound-request risk

A file can be a valid image and a valid archive, or a valid document and a valid script, depending on which parser opens it. Type detection sees the first signature; a downstream tool may act on the second. Re-encoding defeats most polyglots because the output contains only what the decoder understood.

Archive extraction

Archives carry paths, and paths can escape. Reject entries containing traversal segments, absolute paths, symbolic links and device files; cap the entry count, the per-entry decompressed size and the total; and extract into a fresh directory that no other tenant can read.

Scalable vector graphics

An SVG is a document that can carry script and remote references, not an image in the sense the rest of your pipeline assumes. Either rasterise it on upload, or sanitise it with a parser-aware allowlist and serve it exclusively from the isolated origin — never inline it into an application page.

Documents that fetch

Office and PDF formats can reference remote resources, embed active content, and — in a server-side converter — reach internal services. Convert with the network disabled, which turns document processing into another instance of server-side request forgery prevention.

Automated Testing & CI Validation 

CASES = [
    ("shell.php.png",      PNG_HEADER + b"<?php system($_GET['c']); ?>", "accepted, re-encoded, php stripped"),
    ("payload.svg",        b"<svg onload=alert(1)>",                     "rejected or rasterised"),
    ("traversal.zip",      zip_with_entry("../../etc/passwd"),           "rejected"),
    ("bomb.zip",           zip_bomb(ratio=1000),                         "rejected on decompressed size"),
    ("polyglot.gif",       GIF_HEADER + ZIP_CENTRAL_DIRECTORY,           "re-encoded, archive gone"),
]

@pytest.mark.parametrize("name,data,expectation", CASES)
def test_upload_defences(client, name, data, expectation):
    res = client.post("/api/uploads", files={"file": (name, data)})
    assert res.status_code in (201, 400)
    if res.status_code == 201:
        stored = fetch(res.json()["url"])
        assert b"<?php" not in stored and b"onload" not in stored
        assert stored_headers(res)["X-Content-Type-Options"] == "nosniff"

Add a static check that fails the build when an unsafe deserialization call appears anywhere in the tree:

- name: Forbid unsafe deserialization
  run: |
    semgrep --error --config p/security-audit \
      --include='**/*.py' --include='**/*.js' --include='**/*.ts' \
      --metrics=off .

Compliance Mapping

Framework Control Satisfied By
SOC 2 CC6.1 — logical access Signed URLs and tenant-scoped keys on the isolated content origin
SOC 2 CC7.1 — vulnerability detection Upload defence suite and dependency scanning of parsing libraries
OWASP ASVS V12.1–V12.5 — file handling Content-based validation, generated names, non-served storage, explicit types
OWASP ASVS V5.5 — deserialization Value-only formats with schema validation; no object-graph parsing of external input
NIST SP 800-53 SI-10 — information input validation Allowlist plus re-encoding at the boundary
ISO 27001 A.8.28 — secure coding Static gate forbidding unsafe deserialization calls

Common Pitfalls Checklist

Frequently Asked Questions

Is checking the file extension and content type ever enough?

No. Both are supplied by the client, and neither describes the bytes. A file called photo.png with an image content type can contain anything at all, and downstream tools regularly disregard the declared type in favour of what the content resembles. Detect the type from the leading bytes, re-encode where the format allows it, and keep the extension purely as presentation metadata.

Does an antivirus scan make uploads safe?

It helps against commodity malware in files that staff will later download, and it does nothing for the failures that matter here. A vector graphic carrying script, an archive with traversal entries, and a document crafted to trigger a parser bug are all clean to a signature scanner. Scanning is a complement to type validation, isolated storage and an isolated serving origin — never a substitute for them.

Why is deserialization grouped with file upload?

Because both accept opaque bytes from an untrusted party and hand them to something that interprets structure. An archive that writes outside its extraction directory and a serialized payload that names a class to construct are the same failure at different layers: external data chose the code path. The controls rhyme as well — narrow the accepted format, settle structure before acting on content, and never let the payload name the handler.

What size and rate limits should uploads have?

Enforce a maximum body size at the edge as well as in the application, cap concurrent uploads per subject, and bound decompressed size for anything you expand. The edge limit stops the request before it consumes application memory, the per-subject cap stops a single account from monopolising the pipeline, and the decompression bound stops an archive that is tiny on the wire and enormous once expanded. A limit that exists only in application code has already let the bytes arrive.