Group
  • Database

    Backfill a big table without taking my app down

    Plans and writes a batched, resumable backfill for populating a new column on a large production table.

    • Plan & validate
    • Ship faster
    • Advanced
    • General

Free Prompt

I need to add a new column to a large production table and backfill values for existing rows. Write me a safe backfill plan and the script to execute it. Requirements for the backfill: 1. Batch: process rows in small batches (a few hundred to a few thousand at a time) keyed by primary key ranges, not OFFSET, so each batch is fast and the table is never locked for long. 2. Resumable: track progress (last processed ID) so the script can be killed and restarted without redoing work or double-writing rows. Make each batch idempotent. 3. Throttled: add a small delay or row-per-second limit between batches so the backfill never competes with production traffic. It should be safe to run during the day. 4. Observable: log progress per batch (rows done, estimated remaining) so I can watch it. 5. Compatible: the deploy order matters. The new column must be nullable (or have a default) so old and new code both work while the backfill runs. Application code should read the new column with a fallback to the old value until the backfill completes, then we tighten it in a later migration. Do not write a single UPDATE that touches the whole table at once; that is the outage we are avoiding. Do not add a NOT NULL constraint until after the backfill finishes and I confirm it. Give me: the migration for the column, the backfill script, a dry-run mode that reports how many rows need filling without writing, and the follow-up migration to enforce constraints once the data is in.

What This Does / How This Helps

Produces a batched, resumable, throttled backfill script for populating a new column on a table that is too big to update in one shot. The naive backfill is one UPDATE statement. On a small table it is instant. On a real table it locks writes, balloons the transaction log, and your app starts timing out while support email piles up. This is a rite of passage for every growing app and it is completely avoidable. The prompt gives you the full safe pattern: small key-ranged batches, resumable progress, a throttle so production traffic wins, a dry run so you know the scope before you start, and the deploy ordering so old and new code coexist while the data catches up.

Want to skip doing this by hand?

Fortivibe audits your app for all of the areas these prompts cover (and more).

See What We Check

Related Prompts