Group
  • Database

    Make large data exports not kill the server

    Moves CSV/JSON exports out of request handlers into streamed, background-generated downloads.

    • Improve quality
    • Debug & fix
    • Intermediate
    • General

Free Prompt

Fix how my app handles large data exports (CSV downloads, full-account data exports, admin reports). Find every export path and check what happens when it covers 100,000 rows instead of 100. Apply the right pattern per size: small exports (bounded, a few thousand rows) can stay synchronous with a query timeout; anything larger moves to a background job that generates the file in chunks (cursor or keyset iteration, never loading the whole result into memory), writes it to storage, and gives the user a download link or an email when it's ready. Stream responses where synchronous generation stays: write rows as they're read instead of building one giant string. Cap export size and rate limit export requests per user, since exports are expensive and abusable. Don't leave the current 'load everything and stringify it' approach in place behind a loading spinner; that's the thing being fixed. Don't break the file format users already get; same columns, same encoding, and make sure values with commas and quotes escape correctly. Deliver: the per-export pattern chosen, the streaming or background implementation, the size caps, and verification: run an export over a seeded 100,000-row dataset and confirm memory stays flat and the file is complete and well-formed.

What This Does / How This Helps

Moves big exports out of request handlers and into streamed or background generation, so an export over real data can't eat all your memory or hang the request. Export features are built against test data and die on production data. Loading 100,000 rows into memory to build one string is how a single click takes down the server, and it's also a self-service denial-of-service button if there's no rate limit. GDPR-style account exports make this non-optional for many apps: users have a right to their data, and the endpoint that provides it has to actually work at their data volume.

Want to skip doing this by hand?

Fortivibe audits your app for all of the areas these prompts cover (and more).

See What We Check

Related Prompts