Knowledge as Software
Today an AI agent found a production data corruption bug without anyone telling it where to look.
Three players were invisible on a tournament management page. 53 registered, only 50 showing. No error message. No stack trace. Just silence.
We fed the agent raw database tables — no JOINs, no pre-processing, no hints about what might be wrong. It cross-referenced the data, identified the three players by name, explained that their records pointed to a non-existent foursome assignment, and proposed the correct fix.
On the first pass, it identified the three affected players and proposed the correct fix.
But the interesting part isn't the agent. It's what we're building toward.
We manage golf events for leagues across the country. When something doesn't look right — a missing player, a wrong score, a handicap that seems off — we get a phone call. Every time.
We don't want those phone calls anymore. We want the admin to ask the question and get the answer themselves.
So we stopped writing prompts and started writing engineering knowledge.
We call them "steering files" — version-controlled documents that capture domain knowledge accumulated over years of trial and error. How the data model works. What valid state looks like. What each failure mode means. What the fix is.
They live in the repository. They get reviewed. They improve every time we solve a new class of problem.
We don't train the model. We train the organization.
→ Bug found. Root cause understood. → Steering file updated with the new failure mode and fix. → Future diagnosis becomes automatic. → That knowledge never leaves the company.
Every production issue we solve isn't just a fix anymore. It's a new capability that becomes part of the system's institutional knowledge.
For this problem, we didn't need fine-tuning, a vector database, or a RAG pipeline. We needed accurate domain knowledge and the right context.
The goal is a self-correcting system. An admin asks "Why am I missing 3 players?" and gets the same answer an engineer would give — in seconds, without waiting for a support ticket.
That's how expertise scales. Not by hiring more engineers. By capturing what engineers know and making it available to everyone who needs it.
#AIEngineering #SoftwareArchitecture #KnowledgeEngineering #BuildingThingsThatWork #CustomSoftware

