# Profile, Generate, Prove: A method for production-faithful synthetic data

> In part 1, I explained why hand-written test scenarios kept our tests green while delivery dates slipped, and why the answer is data that carries the shape of production by construction. This post describes the method in three steps: profile, generate, prove.

Source: https://www.bluewise.fr/a-method-for-production-faithful-synthetic  
Author: Marc Morel  
Published: 2026-03-12  
Category: Engineering  
Series: Synthetic Data Factory, part 2

One rule runs through all three. **No production row ever leaves the production perimeter; only reviewed aggregates do.** Every technique below is chosen so that it works from those aggregates alone.

## Step 1: Profile production, safely

Everything starts with a statistical portrait of production, computed where the data lives. The profiler is a scheduled job running inside the production perimeter. It reads, aggregates and exports only a profile.

What the profile captures, from the simplest to the most valuable:

1. **Marginal distributions.** The age pyramid of customers, the mix of product categories, the split between channels, amount distributions per transaction type. Categorical fields become frequency tables; continuous fields become binned histograms or fitted parametric laws.
2. **Joint distributions and correlations.** Age alone is not enough: the age pyramid of premium-card holders differs from that of entry-level accounts. Capture the pairs and triples that matter for your workload, such as age × segment, region × category or tenure × balance band, plus a correlation matrix for continuous fields.
3. **Cardinalities and skew.** Products per customer, transactions per account per day, the share of traffic carried by the top 1% of merchants. Skew creates hot partitions, and hot partitions are what performance tests exist to find.
4. **Temporal patterns.** Daily and weekly seasonality, month-end peaks, payday spikes, inter-arrival times between events.
5. **State-transition frequencies.** How often an account moves from opened to active, from active to dormant, from dormant to closed, and how long each step takes.

### Privacy guardrails

Aggregates are safer than rows, but not automatically safe. Rare combinations can still point to a single person. Apply these rules before any profile crosses the perimeter:

- **Minimum cell size.** Suppress or merge any bucket below a threshold agreed with your data protection officer.
- **Coarsening.** Bin ages into bands, round amounts, truncate postcodes to the level your test actually needs.
- **Noise where it matters.** For sensitive joint tables, add calibrated noise (differential privacy). Libraries such as OpenDP make this practical.
- **Allow-listed fields only.** The profiler exports what the specification declares, nothing else. Free-text columns are never profiled verbatim.

The profile itself is a versioned artefact: a set of JSON or Parquet files, reviewed, signed and stored alongside the generator code. When the market shifts, you refresh the profile, not the code.

## Step 2: Generate with three cooperating layers

Generation is not one model. It is three layers, each doing what it does best, wired together by a single declarative specification.

### Layer 1: the statistical core

This layer samples entity attributes so that marginals and correlations match the profile. Because only aggregates leave the perimeter, the techniques must work from aggregates, not rows:

- **Conditional sampling chains.** Sample segment first, then age band given segment, then region given age band, each step read from a profiled table. Simple, fast and transparent to reviewers.
- **Gaussian copulas.** They need only each variable's marginal distribution and a correlation matrix, both of which the profile already exports. They preserve correlations between continuous fields such as age, income band and balance.
- **Marginal-based graphical models.** When dependencies form a graph rather than a chain, fit a model to the exported joint tables. Methods such as PrivBayes, MST or Private-PGM were designed for exactly this: synthesising data from noisy marginals.

Row-trained libraries such as SDV remain an option, but only inside the perimeter. Train there, ideally with differential privacy, and let the fitted model cross the perimeter only after the same review as any profile.

Run this layer on a columnar, vectorised stack such as Polars or DuckDB. Generating attributes in batches of millions is far faster than row-by-row code.

### Layer 2: lifecycle simulation

This is the layer that eliminates inconsistent states. Instead of generating each table independently, the engine simulates entities through time.

Each business entity (customer, account, card, contract) is described by a state machine: states, allowed transitions, guards and timing distributions taken from the profile. The simulator creates a customer, opens an account after a sampled delay, issues a card, emits transactions and may later close the account. Simulated time advances in memory, so years of history take seconds. Because every record is the consequence of a legal transition, foreign keys, timestamps and statuses are consistent by construction.

On top of the state machines, a set of **invariants** acts as a safety net: a closed account carries no new transactions, a refund never exceeds its payment, a balance equals the sum of its movements. Any record that violates an invariant fails the run.

### Layer 3: LLM-authored rules and dictionaries

The LLM contributes at design time only, and everything it produces is reviewed before the engine uses it:

- **Rule extraction.** Feed the LLM your DDL, enum definitions, validation code and API contracts. Ask it to draft the state machines and invariants in your specification format. Engineers review and merge them like any pull request.
- **Seed dictionaries.** The LLM curates the values the engine samples from: company names, merchant labels, and templates for free-text fields such as transfer references or support notes. The next section details them.
- **Edge-case catalogues.** Ask the LLM which unusual but legal situations your domain allows: joint accounts, minors with a guardian, cross-border customers, accounts reopened after closure. Each becomes a scenario with an explicit, profile-driven frequency. This is precisely the diversity our hand-written scenarios never reached.

At runtime, there are no LLM calls. Free text is assembled from reviewed templates and dictionaries, which keeps every run fast, cheap and reproducible.

### The specification ties it together

A single declarative file (YAML or a typed DSL) describes entities, the profile fields each attribute samples from, the dictionaries, the state machines, the invariants and the target volumes. It is the contract between the profile, the LLM-authored rules and the engine. Keep it in Git, review it, and treat changes to it like changes to production code.

## Seed dictionaries: the Faker idea, grounded in your market

Statistics tell the engine how often a value appears; seed dictionaries tell it which values exist. They are the curated lists every realistic record draws from: email providers, first and last names, company names, cities, street names. (They are not to be confused with the random seed that makes each run reproducible.) If you have used Faker in Python or Datafaker in Java, you know the idea. The difference is that every value is paired with a weight measured on your own market, not picked uniformly from a generic locale.

### Consumer email providers

The provider mix is national: gmail.com, orange.fr, free.fr and hotmail.com dominate in France; gmx.de, web.de and t-online.de in Germany. The skew matters, because domain columns feed indexes, partitions and fraud rules. Values come from Faker locales and public lists, completed by the LLM for the long tail. Weights come from the profile's per-country domain share, with rare domains merged into an "other" bucket. Local parts are built from the customer's synthetic name, using profiled patterns (firstname.lastname, f.lastname, name plus digits).

### First and last names, per country and script

Names come from per-locale dictionaries, weighted by country and birth year. Their structure follows local conventions:

- **Order and form.** Family name first in China, Japan, Korea and Hungary; two surnames in Spain and Portugal; patronymics in Russia; single names in parts of Indonesia.
- **Scripts.** Customers from non-Latin locales carry the native form (王伟, Иван Петров, 佐藤花子, محمد العلي) and its standard transliteration (Pinyin, ISO 9, Hepburn). Emails use the transliterated form, unless you test internationalised addresses.
- **Hard characters.** Diacritics (é, ß, ł), apostrophes and hyphens (O'Brien, Jean-Pierre) appear at their real frequency.

This is where realistic names pay off: they exercise encoding, collation, sorting, search and the validators that reject them in production.

### Business names and professional emails

The LLM generates fictional company names per sector and country, with legal forms (SAS, GmbH, Ltd), filtered against open registries such as SIRENE. Each company gets a corporate domain and addresses: firstname.lastname@company.fr for employees, plus contact@ or invoices@. Profiled company size sets the head count, from three addresses to hundreds.

### Consistency with the rest of the record

Seeds become truly realistic when they agree with the entity they decorate:

- **First names follow the age pyramid.** Popular first names change by decade. Open datasets, such as the INSEE first-name file in France, give frequencies per birth year, so a customer born in 1962 gets a name that was common in 1962.
- **Providers follow geography.** A customer living in Germany draws from the German provider mix, not the French one.
- **Addresses are real places.** Street names, postcodes and cities come from open address data, such as the Base Adresse Nationale in France, so geocoding and postcode validation behave as in production.
- **Uniqueness is an invariant.** Two customers must never share an email. Collisions are resolved deterministically, with a numeric suffix derived from the random seed, and the consistency gate checks it.

### A word of caution: real domains reach real people

A synthetic address on gmail.com may well belong to an actual person. If a test environment ever sends a notification, that person receives it. Route every outbound email from non-production environments to a capture sink such as Mailpit, and block external SMTP at the network level. When a test does not need a realistic domain mix, prefer the reserved domain example.com or the .test extension (RFC 2606).

### What a dictionary looks like

Each dictionary is a small, versioned file listing values, locale, and the source and licence of both values and weights. It is reviewed like code, alongside the profile and the spec.

```yaml
dictionary: consumer_email_providers
version: 2026.10.1
locale: fr_FR
weights_from: profile/email_domain_share_fr.parquet
values_from: [faker:fr_FR, public-provider-list, llm-curated]
entries:
  - domain: gmail.com
  - domain: orange.fr
  - domain: free.fr
  - domain: hotmail.com
  - domain: laposte.net
other_bucket: public-provider-list
```

The weights are deliberately absent from the file: they live in the profile and are joined at generation time. Refreshing the market mix is then a profile update, with no change to the dictionary.

## Step 3: Prove it before you trust it

A synthetic dataset is a deliverable, so it gets a quality gate. Every generation run is checked against the profile it came from, and the run fails if any gate fails. This is the opposite of our old, easily green tests: here the bar is set by production, not by our imagination.

| Gate | What it checks | Typical method |
| --- | --- | --- |
| Marginal fidelity | Each column matches its profiled distribution | Kolmogorov–Smirnov test for continuous fields, total variation distance for categorical ones |
| Joint fidelity | Key correlations and cross-tabs are preserved | Correlation matrix delta; comparison with the joint tables in the profile |
| Skew and cardinality | Hot keys and fan-outs look like production | Top-N share, percentiles of children per parent |
| Consistency | No invariant or state-machine rule is violated | SQL assertions over the full dataset (DuckDB works well) |
| Privacy | No output reproduces a combination rarer than the minimum cell size | Generation uses vetted aggregates only, so this is a check of the output against the profile, not a comparison with real rows |
| Workload realism | The system behaves as it does in production | Cache hit ratio, query plans and partition heat on a smoke run, compared with production telemetry |

The last gate is the one teams skip and later regret. Statistical fidelity is necessary, but the real test is behavioural: does the database choose the same plans, does the cache hit at the same rate, do the same partitions run hot? Production telemetry contains no customer data, so this comparison stays within the privacy rule.

When a gate fails, the LLM earns its keep a second time. Give it the failing report, the specification and the relevant rule, and ask it to explain the gap and propose a fix. An engineer still approves the change, but the loop shrinks from a day of investigation to minutes.

## What changes

> The bar is set by production, not by our imagination

Hand-written scenarios test what we thought of. This method tests what production actually does. Each step moves a decision from someone's intuition to an artefact that can be reviewed:

- **Profile.** "What does production look like?" stops being a guess. It becomes a versioned set of aggregates, computed inside the perimeter and checked for privacy before it leaves.
- **Generate.** Realism stops being hand-crafted. Statistics decide how often a value appears, seed dictionaries decide which values exist, and lifecycle simulation makes every record the result of a legal transition. Inconsistent states disappear by construction instead of being chased one bug at a time.
- **Prove.** A green run stops being a matter of faith. Every dataset passes the same fidelity, consistency, privacy and workload gates, and a failing gate fails the run.

The LLM's role changes too. It no longer invents test data; it drafts rules, curates dictionaries and lists edge cases at design time, and explains failing gates afterwards. Engineers review everything it produces, and no model is called at runtime, so every run stays fast, cheap and reproducible.

What you maintain changes as well. Instead of a growing pile of fixtures, you keep three things in Git: a profile, a specification and a set of dictionaries. When the market shifts, you refresh the profile, not the code. And throughout, one rule holds: no production row ever leaves the production perimeter.