The Architecture of AI That Actually Works
[This is the second in a 3-article series that draws on a conversation with Andrew Amann, CEO of NineTwoThree AI Studio, on the CollabTalk Podcast.]
If the first question in any AI initiative should be “where does work get stuck?”, the second question is: “what kind of answer does this problem actually need?” That distinction, between problems that require precise, deterministic answers and problems that benefit from intelligent, probabilistic reasoning, is one of the most important architectural decisions in AI deployment. And most organizations never make it consciously.
Two Types of Problems, Two Types of Systems
Deterministic problems have fixed correct answers. The price on a contract. The routing status of a shipment. The specific product recommendation your company has reviewed and stands behind. These answers don’t benefit from a language model’s creative interpretation. They require accuracy. The data exists. You just need to retrieve it reliably.
Probabilistic problems are different. Summarizing a complex document. Drafting a first version of a communication. Routing a nuanced customer inquiry to the right team. These are tasks where a language model’s ability to reason across context, recognize intent, and generate coherent output is genuinely valuable.
The failure mode most AI projects encounter is treating all problems as probabilistic: letting a language model handle questions it was never designed to answer reliably. The solution is architecture: systems that route each question to the right tool based on what kind of answer it requires.
Agent Orchestration: Where Trust Gets Built
The shift that has fundamentally changed AI reliability in enterprise settings is the emergence of agent orchestration — multi-agent systems where specialized components handle specific types of tasks, coordinated by an orchestration layer that routes work intelligently.
Think of it this way: a logistics company dealing with a hurricane disruption doesn’t need a language model to guess at rerouting. It needs a system that can write a precise database query to pull current routing data, retrieve the accurate answer, and then present it conversationally. That combination, with structured precision on the backend and natural language on the front end, is where enterprise AI actually earns trust.
“Ten percent of it is always started by a human and ten percent of it is ended by a human. Eighty percent of it should be done by the AI in the middle — those concepts are how we consider cost, accuracy, and latency.” — Andrew Amann, CEO of NineTwoThree AI Studio
The QA Problem Nobody Talks About
Even well-architected AI systems require something most organizations underinvest in: quality assurance. Not just at launch, but continuously. Who is checking whether the answers being returned are accurate? Who is reviewing edge cases? Who owns the feedback loop when the system gets something wrong?
This is the unsexy part of AI deployment, and it’s where a lot of otherwise-promising projects quietly degrade. Language models can drift. The data they draw from goes stale. New scenarios arise that weren’t anticipated. Without an active QA practice and a clear owner, even a well-built system eventually becomes a liability.
The organizations getting consistent ROI from AI treat QA as a standing function, not a launch activity. They build in mechanisms for flagging uncertain answers, routing edge cases to humans, and continuously updating the system’s knowledge base. The model is never “finished.” It’s a team member who needs ongoing management.
Model Agnosticism as a Strategic Asset
One more architectural principle worth building in from the start: don’t marry your system to a single model provider. The landscape is moving too fast. Organizations that build model-agnostic infrastructure — where the underlying language model can be swapped without rebuilding the entire system — preserve optionality as better, cheaper, or more specialized models emerge. That flexibility compounds in value over time, and it’s far easier to build in at the start than to retrofit later.
Questions to consider:
- For your highest-priority AI use case: does it require a deterministic answer, a probabilistic one, or a combination, and is your current architecture designed for that distinction?
- Who owns QA for your AI systems after launch, and what does their process actually look like?
- If a better model became available tomorrow, how hard would it be for your organization to switch?
▶ Next in the series → Part 3: How to Scale AI Across Your Organization — from first win to compound competitive advantage.


