Subscription businesses lose money in three ways that look different and are really the same: a card fails, a checkout is abandoned, an invoice goes past due. Recoup works all three through one loop. It watches for revenue at risk, works out why each payment failed, prices the cheapest intervention likely to work, executes it inside hard compliance and stopping rules, and then measures what that was actually worth.
There is no language model anywhere in it. Message copy comes from reviewed templates. The intelligence is in the decision policy and an uplift model, not in generation, and a given seed replays byte for byte.
What one failed payment goes through
A signed webhook opens a case, and what is owed is derived from ledger events; no balance is ever stored. The bank’s error code maps to one of eight recovery classes. A playbook then prices every available action against doing nothing, and doing nothing frequently wins.
Before anything is sent, 35 compliance rules and 18 stop conditions run as ordinary code. They run again immediately before execution, and a rule that throws counts as a refusal, so a bug in a rule fails closed instead of quietly letting a message out. No prediction can talk its way past them; the bounds live outside the model on purpose.
Cohort health runs alongside. When a particular payment method and issuer combination starts failing, retries into that route pause and customers stop being messaged about a failure that belongs to us or to the bank, not to them.
Measuring it honestly
This is the part I would defend first. Most recovery tooling reports gross amount recovered, which is close to meaningless: most overdue payments arrive eventually whether you chase or not, so any tool that chases can take credit for them.
Recoup holds one case in five entirely out of its own reach, runs the old fixed schedule on those, and reports the difference between the two arms with a confidence interval. The estimate is post-stratified on the amount band and failure class the arms were assigned on, which is the same quantity as a plain difference of means with roughly 28% less variance.
Because every decision also logs the probability with which it was taken, a policy the agent never ran can be scored from the log alone. That report compares five policies under IPS, SNIPS and a doubly-robust estimator, and prints the overlap and effective sample size next to each one. Right now “always send a WhatsApp nudge” tops the table on an effective sample of about 20 cases, so the report says plainly that this is an artefact and not a finding.
What the numbers actually say
Out of sample, on a seed the model never saw, incremental recovery came out at +1.62pp with a 95% interval of [-3.09, 5.88]. That interval contains zero. It is not a win.
The baseline is not a straw man either: three retries, one SMS and one email is roughly what a competent merchant already does. The agent contacts about a third as often and lands in the same place. Parity at a third of the contact volume is a real result, and it is not the result anyone hopes for.
The ablations are more informative than the headline, because comparing configurations against the same simulated world removes the variance between worlds:
| Layer removed | Change in recovered fraction | Earns its place |
|---|---|---|
| Policy gate | -9.69pp | No, it costs money |
| Incumbent floor | +6.27pp | Yes |
| Timing, diagnosis, uplift, reviewer, allocation, action gate | within noise | Not detectable |
One layer is significant across every run, and it is the floor that stops the agent regressing below the fixed schedule. The rest are reported as undetectable and left in, because that is what the intervals say. Alongside that: zero policy violations, propensity recorded on 100% of decisions, and a hash-chained audit log that CI checks still verifies.
The model
Training runs in Python with scikit-learn; serving runs in TypeScript. The boundary between them is a committed JSON file, so a clone with no Python still runs the demo, the tests and the measurement.
Six candidates are compared under grouped cross-validation that holds whole cases out. Selection uses the one-standard-error rule, taking the simplest candidate within one standard error of the best, because cross-validated Qini is itself an estimate and chasing its maximum just selects for a lucky fold split. The winner is an S-learner over gradient boosting at depth 2, at Qini 0.171 ± 0.011.
Three things keep the language split honest. TypeScript owns feature encoding and Python only composes blocks it is handed, so train and serve cannot skew. The exporter re-implements the TypeScript scorer and refuses to write a model whose replayed score differs by more than 1e-9. And 64 encoded rows with their scores are committed as golden vectors, re-derived by both the TypeScript suite and CI.
The per-action results are the part worth arguing about. Cross-validated Qini ranges from +0.230 on retrying the charge down to -0.443 on mandate repair, which is considerably worse than random. An action is only allowed to be ranked when its own Qini clears its own standard error, which leaves 5 of 9 trusted and hands the other four back to the playbook. One action sits at +0.041 ± 0.070 and is excluded despite a positive point estimate.
Constraints the build runs under
Date.now, new Date, Math.random and randomUUID all fail lint. The engine
takes an injected clock, a seeded RNG and an ID factory instead, which is what
makes a seed replay identically. Money is integer paise behind a branded type.
Outstanding balances are always derived, never stored. Anything with a threshold,
cap or rate lives in YAML, not in code, so changing policy does not mean changing
the engine.
There is one composition root. The webhook route and the simulation harness both
call it, so the thing under test is the thing that runs. DRY_RUN defaults to
true and turning it off takes two independent signals, so the system can run
against real traffic in observation mode before it is allowed to send anything.
102 TypeScript source files, 622 tests across 37 files, property-based tests via fast-check.
Honest limits
All the data is simulated, and the simulation constants are written down in the repository where they can be argued with, not buried in code. The compliance rules are engineering rules written against public regulatory sources, not legal advice. The uplift machinery is separately validated against the Hillstrom email trial, a real randomised experiment over 42,694 customers, but that tests the statistics and not the payments domain, and I keep those two claims apart deliberately.
