How we cut compliance costs by $2.4M with AI

When Meridian Financial came to us, their compliance team was spending 60% of their time reading regulatory documents and manually tagging obligation types. The brief was simple: automate the classification work, keep humans in the loop for edge cases, and hit 90%+ accuracy.

Scoping: defining what 90% actually means

We spent two weeks mapping their classification taxonomy. Fourteen obligation types — but only six accounted for 85% of the volume. We agreed to optimise for those six first. A model that's 95% accurate on high-volume obligations and routes everything else for human review is more valuable than one that's 80% accurate on everything.

Architecture: LangChain + GPT-4o classification

The pipeline has three stages: ingestion converts documents to clean text, classification runs a two-step embedding search then GPT-4o call, routing sends high-confidence results to the compliance system and low-confidence ones to a human review queue with the model's reasoning attached.

What we learned

The hardest part wasn't the model. It was the document parsing. Regulatory documents have inconsistent formatting, tables that span pages, footnotes that modify the meaning of paragraphs above them. We wrote more parsing code than classification code.

Six months post-deployment: $2.4M in annual cost reduction, 94% accuracy on the six priority obligation types, and a compliance team that describes the tool as "the first AI thing that actually works."