Orchestration inside, choreography between: Combining Temporal and Kafka

A customer switching banks makes one decision. The bank inherits days of work: redirect every recurring direct debit, sweep the balances out, close the accounts in the right order, and keep the customer informed while creditors and counterparty banks answer on their own schedule.

It is the textbook case for durable execution. It also shows why orchestration and choreography are not rival styles to choose between, but two layers of the same design — one inside each process, one between them.

The mandate is the parent

The parent workflow owns the switching mandate: the customer, the statutory deadline, the target end state. Its workflow ID is the business key, switch-{mandateId}. That single choice buys idempotency for free: whatever triggers the switch twice, Temporal refuses the second start.

The parent encodes the order that matters. Direct debits move first, balances are swept next, accounts close last. Closing an account before a pending debit has been redirected is how a customer ends up with a rejected rent payment. The sequence lives in code, not in a runbook. When a step fails for good, the parent compensates in reverse: a sweep bounced by the receiving bank is credited back before anything is closed.

Why children, not activities

Each direct debit migration and each account closure runs as a child workflow, switch-{mandateId}/debit-{ref}, for three reasons.

First, history. Every event lands in the workflow history, and history is bounded. A customer with forty direct debits, each waiting on retries and acknowledgments, would push a single workflow toward its limits. Children partition the history per business object.

Second, failure isolation. A creditor that never answers keeps one child waiting, not the whole mandate. The parent sees an explicit, typed outcome per debit and decides: escalate, notify the customer, or proceed without it.

Third, versioning. Long-running workflows replay their original code. Splitting by business object means a change to the debit logic only affects children started after deployment, while in-flight children finish on the version they began with.

The parent close policy is a deliberate choice. REQUEST_CANCEL lets a customer who withdraws the mandate trigger a clean unwinding: each child receives the cancellation and runs its own compensation instead of being killed mid-transfer.

Waiting, done properly

Most of a switch is waiting. Durable timers express deadlines directly: wait for the creditor, but no longer than the statutory window, then escalate.

External answers arrive two ways. A counterparty bank acknowledgment is a fact the workflow reacts to: a signal, fire-and-forget. A customer confirming the closure date needs an answer back: an update, validated by the workflow and returning a result synchronously to the calling API. Using signals for everything forces the frontend to poll; using updates for everything couples partners to your latency.

After closure, one small workflow keeps watching: debits still hit the old account for months. A sentinel forwards them and uses continue-as-new periodically to keep its history short.

Nexus: crossing team boundaries

Transfers belong to the payments team, with their own namespace, workers and release cadence. The switching workflow should not know their task queues or workflow IDs.

This is still orchestration, only across a team boundary: the mandate owns the outcome of the transfer and must react to it, so an event thrown over the wall would not do. Nexus gives that boundary a typed contract. Payments exposes a service with an initiateTransfer operation behind a Nexus endpoint. The switching workflow calls it like any operation; the handler starts a workflow in the payments namespace; the result, or the cancellation, flows back across the boundary. Each team keeps its operational ownership, and the contract is what they version. Idempotency crosses the boundary too: the request carries a key derived from the child’s workflow ID, so a retried call never becomes a second transfer.

Kafka: facts in, facts out

Temporal owns process state; Kafka carries facts. Asking Kafka to orchestrate is where most event-driven sagas go wrong: retry topics, dead-letter queues and state scattered across consumers become an implicit workflow nobody can read.

Inbound, a consumer turns each relevant event into a signal-with-start against the deterministic workflow ID. Kafka delivers at least once, so this call must be idempotent, and the offset is committed only after Temporal accepts it. Partitioning by mandate ID preserves ordering where it matters.

Outbound, a workflow never publishes directly: it would break determinism. Publishing is an activity, and activities retry, so duplicates will happen. Kafka’s idempotent producer does not cover a retried activity. Each event therefore carries an ID derived from the workflow ID and the step, and consumers deduplicate on it.

Two layers, one design

The two styles meet at three hinges, each a deliberate handoff.

Choreography starts orchestration. The switch request is a fact published by the onboarding domain. A consumer turns it into a signal-with-start, and from that moment one workflow owns the mandate.

Orchestration feeds choreography. When the mandate completes, it publishes MandateCompleted. Reporting, marketing and the data platform react without the mandate knowing they exist.

Orchestration can cross a boundary when one party owns the outcome, as with Nexus above. Otherwise, a fact is enough.

If a process has an owner, a deadline and an end state, orchestrate it. If it is a fact others may care about, publish it and let them choreograph. Every orchestrated process is a node in a wider choreography, and each node is readable on its own.

What changes

• For the customer: one decision, then visible progress. A silent creditor becomes an escalation with a deadline, not a rejected payment on next month’s statement.
• For engineering teams: the process lives in code a newcomer can read, and each team owns its side of a contract: typed operations through Nexus, facts on Kafka for everyone else.
• For operations and compliance: every mandate has a state you can query, a failure you can resume from, and an audit trail that is the execution itself, not a log reconstructed after the fact.

A bank switch used to be a nightly batch and an apology. Built this way, it is something you can watch, resume, and explain line by line to a regulator.