The Checklist That Checks Itself

A loan file is ready to go. Everything is in it: identification, income evidence, statements, the signed forms, the notes from three phone calls. Before it can be submitted, someone has to sit down with the regulator's checklist and work through it item by item.

Not all of the items apply. Which ones do depends on the borrower. A self-employed applicant pulls in requirements a salaried one does not. A refinance asks different questions to a purchase. Add a guarantor and a new set appears. So the first job is not checking the file, it is working out which parts of the checklist are even in play, and that decision lives in the head of whoever happens to be doing it.

Then comes the checking. Is there a document that covers this obligation? Is it the right kind? Is it recent enough? Has anyone actually reviewed it, or is it just sitting in the folder? And when it is all done, someone opens the document list and picks out, one by one, the files that back each item, so the bundle that goes to the lender contains the evidence and not the whole drawer.

We built an engine that does the recall and the arithmetic, and leaves the judgement to a person. The client is a consumer credit licensee. The checklist runs to twenty-one logical items.

The Checklist Tax

The thing that made this expensive was not any single step. It was that every step had to happen every time, and none of it left a trace.

Applying the checklist by hand is roughly an hour of a compliance officer's attention per file, and it is the kind of hour that cannot be interrupted, because the state of the exercise is held in working memory. Document currency is worse, because it is arithmetic dressed up as a judgement call: work out the document's date, work out today, work out the gap, compare it to a window you are meant to remember. Do that a dozen times per file and you will eventually get one wrong, and the one you get wrong will not announce itself.

The absence of a trace is the part that really costs. When a file is questioned six months later, the question is not just "was this compliant". It is "who decided it was, on what evidence, and when". A checklist worked through in someone's head produces a submitted file and nothing else. Every reconstruction after the fact is archaeology.

That question is being asked more often, of fewer people. ASIC's Report 825, published November 2025, shows 4,466 approved credit licences on foot at 30 June 2025, down from 5,503 seven years earlier, with 104 new licences granted in the 2024-25 year against 273 in 2017-18. Only 63% of finalised applications were granted. Meanwhile ASIC's annual report for the same year describes a 50% increase in investigations and 829 completed targeted surveillances, though those are regulator-wide figures rather than credit-specific ones. A shrinking population of licensees, each carrying more scrutiny, is a bad environment in which to be holding your compliance reasoning in your head.

Rules as Data, Not Code

The first decision set the shape of everything after it. The checklist is not written in TypeScript. It is a JSON document, and the code is a generic interpreter that walks it.

Each item declares two things: the conditions under which it applies, and a tree describing what would satisfy it. The tree is made of ANDs, ORs, and leaves. A leaf is either a piece of evidence that would need to be on file, or an attestation, which is a human saying so. Around ninety leaves hang off the twenty-one items, and some items branch by the structure of the borrowing entity, so the shape is not uniform.

Two combinators do the work of collapsing that tree into a verdict. AND takes the worst status of its children. OR takes the best. That is the whole aggregation model, and it fits on a line, which is exactly what you want in the part of a system that has to be defensible.

The payoff is that changing the checklist is a data change. When the regulator or the aggregator revises an item, nobody edits an interpreter. And because the ruleset carries an identifier that doubles as a version stamp, and that stamp is written onto every result, a ledger computed last quarter stays attributable to the rules that were in force last quarter. That matters more than it sounds. A compliance system that silently reinterprets history under today's rules is worse than no system, because it looks authoritative while being wrong.

Deciding What Even Applies

Before an item can be checked, the engine has to decide whether it is in scope for this borrower. It derives a set of signals from the file, the employment shape, the purpose of the loan, the entity type, and matches those against each item's declared conditions.

The interesting part is what happens when it cannot tell. There are three possible answers, not two: it applies, it does not apply, or a person needs to confirm. Every verdict carries a plain-English reason and a tag saying where it came from.

That third answer is the load-bearing design decision in the whole system. The tempting version has two outcomes, and quietly resolves the unknown cases to "not applicable" because that keeps the ledger tidy. It also produces a file that looks clean and is not, which is precisely the failure a compliance tool exists to prevent. So the rule we wrote and kept is that the engine never silently drops an item. If the field that decides applicability is genuinely not captured anywhere in the file, it says so and asks.

A related rule turned out to matter just as much: an absent section is not a determination. If the income section has never been filled in, that is not evidence the borrower has no income. The engine distinguishes "we looked and the answer is no" from "nobody has told us yet", because collapsing those two is how a system produces confident nonsense.

There is one item where the attribute that would decide it is not captured anywhere in the file, in any form. We could have inferred it. Instead that item always routes to a human, and the code says why. Being honest about the boundary is worth more than an extra green tick.

Evidence, and Whether It Still Counts

Matching documents to obligations is deliberately dull. A leaf names the kind of evidence it needs, the engine finds documents of that kind attributed to the right applicant, and a rejected document never matches anything.

Currency is where the arithmetic used to leak. Dates arrive in whatever format the source produced, so parsing is defensive and everything is built in UTC, because a date that drifts by one day across a timezone boundary is a bug you will find at the worst possible moment. Where a leaf declares a window, the document is fresh, stale, or, if no date could be extracted at all, it becomes a question for a person rather than a pass.

Every item declares how much of it can be automated. Some pass on evidence alone. Some need evidence plus a human confirming the compliance aspect, which evidence alone cannot establish. Some are pure sign-off, because no document exists that would prove them. Being explicit about that per item is what stops the system overclaiming. It is not pretending to have read the file. It is telling you which parts it can settle and which parts it cannot.

The engine also never returns a bare pass or fail. There are four statuses, and the two in the middle carry the real information: this needs review, or this needs someone to attest. A binary output would have forced every uncertain case into one of two lies.

The Machine's Opinion and the Human's

Staff sign off individual attestations and can override an item outright with a reason. Both are recorded with who and when, and both are written to the file's activity log as events.

The structural decision underneath is that the human's answer and the machine's answer are stored separately, and both survive. Every item exposes the computed status and the effective status. An override changes what the file reports; it does not erase what the engine concluded. Six months later you can still see that the system said one thing and a named person decided otherwise, on a stated reason, on a date.

Everything is treated as derived state and recomputed from scratch on demand, with one exception: human input is carried forward, and then fed back in as an input to the next evaluation. So a sign-off is not an annotation stapled to the outside of the result. It is part of the calculation.

That purity had an unplanned benefit. Because the entire engine is side-effect free apart from a single persistence layer, we could point it at the live book read-only and evaluate every real file without writing a row anywhere. Every ledger, computed on production data, before the feature was ever switched on. Two passes, no crashes, and a list of the cases it handled badly, which is the only kind of dry run worth doing.

The last piece is the export. The bundle is defined as exactly the documents backing items that both apply and pass. Not the folder, not a selection someone made at 6pm on a Friday. The set falls out of the ledger, which means the thing you send and the thing you claim are the same thing by construction.

By the Numbers

This engine replaces the recurring cost of two operational functions:

• A compliance officer (~$105,000 AUD/year, Robert Half 2026 Financial Services Salary Guide) who would otherwise apply the checklist to every file by hand, decide which items are in scope for each borrower, and hold the reasoning that justifies a submission.
• Broker support, roughly half a role (~$80,000 AUD/year, Hays Salary Guide FY26), who would otherwise verify document currency item by item and assemble each submission bundle document by document.

Operational savings breakdown:

• $145,000 AUD in annual savings, on base salary figures excluding superannuation, so the true number is higher.
• 400 dev hours avoided against what a traditional team would need. The code is around 2,200 lines of engine with another 800 of tests and a ledger interface on top, but the weight is in the analysis: turning a twenty-one item regulatory checklist into a declarative ruleset of roughly ninety satisfaction leaves, with applicability conditions and currency windows attached. This is an estimate of comparable build scope, not a measurement.
• 1.5 FTEs replaced.
• Automation type: workflow.

Eighty test cases cover the combinators, the applicability resolution, the date parsing and the bundle selection, which are the four places a wrong answer would be both plausible and expensive.

What Ambiguity Is Worth

The instinct when automating a checklist is to make it decide. A green tick is satisfying, a page of them more so, and there is real pressure to produce a system that answers.

The engine we shipped declines constantly. It flags applicability it cannot determine. It refuses to read a blank section as a negative. It marks a document that carries no extractable date as a question rather than a pass. On one item it always asks a person, because the deciding fact is not written down anywhere. Each of those is a place where guessing would have produced a cleaner-looking result and a worse file.

What automation is actually good for here is recall and arithmetic, done identically every time, with the working shown. The judgement stays with the person whose name goes on it. What they get back is the hour they used to spend reconstructing the question, and a record that answers it for them the next time somebody asks.