AI Agent Architecture for a 30-Person Team (No DevOps)

When a CEO of a 40-person logistics company asked me what "AI agent architecture" meant for a team his size, I told him the honest answer: most reference architectures floating around were drawn for 5,000-person enterprises with a platform team, and copying them is how SMBs burn six months and a quarter of their budget on plumbing.

Why architecture matters more than the model

The model is a commodity. Anthropic, OpenAI, Google ship comparable Tier-1 models within weeks of each other. What separates an agent that works from one that quietly hallucinates is everything around the model: how context arrives, how decisions are logged, how a human gets pulled in when the agent is unsure.

MIT's 2025 study found 95% of GenAI pilots fail to reach production ROI. The pattern in failed pilots is almost never "the model wasn't smart enough." It's "we couldn't tell what the agent did, so we couldn't trust it, so we shut it off."

Definition: AI agent — a software process that takes a goal, decides intermediate steps using an LLM, executes those steps via tools (APIs, databases, search), and returns a result. Unlike a chatbot, it acts; unlike a script, it adapts.

The four-component reference architecture

For a company of 30-500 employees with no dedicated DevOps, the reference architecture has four parts. That's it.

1. Orchestrator (the brain stem)

This is the layer that takes a request, calls the LLM, calls the tools, and decides the next step. For SMBs, the choice is between a no-code/low-code platform (n8n, Make, Zapier with AI nodes), a hosted agent platform (Lindy, Relay, OpenAI's Assistants API), or a thin custom layer (LangChain, LlamaIndex, or just direct SDK calls).

Definition: Orchestrator — the controller that runs the agent's decision loop. It owns "what to do next" given the current state.

For a 30-person team without engineers, start with hosted no-code. You will outgrow it eventually. That's fine — outgrowing it means the agent is producing value, which is the only outcome that matters.

2. Context store (the memory)

The agent needs to know: who is this customer, what did they ask last week, what's our policy, what's the SKU catalog? In enterprise architectures this becomes a vector database with a RAG pipeline. For an SMB, that's overkill in 80% of cases.

Cheaper alternatives that work:

• System-of-record direct lookup — query Salesforce, HubSpot, or Postgres directly. No vector store needed.
• Document store with embedded search — Notion, Coda, or a Google Drive folder indexed by the orchestrator's built-in search.
• Vector DB only when you need semantic search over unstructured text — call transcripts, support tickets, long PDFs.

Definition: RAG (Retrieval-Augmented Generation) — pattern where the agent retrieves relevant context from a knowledge store before answering. Reduces hallucination rate dramatically when the source data is clean.

3. Human escalation path (the safety net)

The single biggest mistake I see in SMB agent rollouts is launching without a clear "the agent is unsure — what now?" path. Klarna walked back its full-AI customer-service agent in 2025 specifically because escalation was too slow. Stanford's 51-deployment study found escalation-routing agents yield ~71% productivity gain versus ~30% for approval-routing — the architecture pattern matters more than the model.

Three things must exist before launch:

1. A confidence threshold — below X, escalate.
2. A named human owner — not "the team," a specific Slack handle.
3. An SLA — "escalations get touched within 30 minutes during business hours."

4. Observability (the dashboard)

You need to see what the agent did. Not metrics — trace logs. Every prompt, every tool call, every output. For a 30-person team this can be a Postgres table the orchestrator writes to, plus a one-page admin view that shows the last 200 runs.

[Run #1432]  Time: 14:22 UTC  Duration: 4.1s
Goal: "Look up invoice INV-2891 for Acme Co"
Step 1: query_crm(account="Acme") → 1 result
Step 2: query_invoice(id="INV-2891") → status=overdue, 11 days
Step 3: draft_email(tone="firm-but-polite") → 87 words
Result: Draft saved to Olha's inbox, awaiting approval
Confidence: 0.91
Escalated: no

Without this, you cannot debug, you cannot improve, you cannot justify the agent to your CFO when she asks "what is this thing actually doing?"

Team scan (what AI champions report after week 1)

• Adoption inside the pilot team rises fast in the first 5 days, then plateaus around 60% — the plateau is normal.
• The most-cited use case is invoice or document lookup, not creative drafting.
• Average saved time per user lands between 2-4 hours/week after the first 14 days.
• About 15-20% of agent runs hit the escalation path in week 1; this drops to 5-8% by week 4 as prompts and context improve.
• Champions surface the same 3-4 broken edge cases the founder never knew existed (vendor name typos, wrong currency, weekend handoffs).
• Shadow AI usage drops noticeably — people switch from copy-pasting into ChatGPT to using the sanctioned agent.
• Nobody asks "is this AI?" by week 2. It becomes a tool, like email.
• The biggest blocker reported is not the model — it's missing access to one source system that wasn't included in the original scope.

Tool tip (Course for Business): The 30-person team scenario is exactly where the AI Champions (1:15-20) ratio earns its keep — one trained champion per 15-20 staff covers a 30-person company with 1-2 people who can debug agent runs, write better prompts, and onboard the rest. This is more durable than hiring an external agency: the knowledge stays in the company. Our 5-day program trains every employee to ship one automation, and we identify 1-2 champions during week 1 based on who actually builds and ships.

What to skip (for now)

The conversations I have with founders almost always start with the wrong question. They ask "should we use Pinecone or Weaviate?" before they have a single agent in production. So a list of things to defer past month 3:

• Vector database (use direct DB lookup until you have unstructured-text RAG needs)
• Self-hosted LLM (Gartner found CIOs miscalculate AI infrastructure costs by up to 1,000% — usually self-hosting is the trap)
• Custom evaluation framework (use the orchestrator's built-in run logs)
• Multi-agent orchestration (one agent doing one job is plenty for the first quarter)
• Fine-tuning (prompt engineering + RAG covers 95% of SMB use cases)

The Builder.ai $1.3B collapse in 2024 is the cautionary tale: a company that overbuilt the platform layer before proving the agent layer worked. SMBs do not have that runway.

Copy-paste decision template

Before architecting, fill this in. If you cannot answer all 6 in one page, you are not ready to build.

1. Job to be done (one sentence):
   ____________________________________________

2. Who owns the agent (named person, not team):
   ____________________________________________

3. Source systems the agent must read:
   - ____________________
   - ____________________
   - ____________________

4. Source systems the agent may WRITE to:
   - ____________________ (escalation required: y/n)

5. Escalation path:
   - Trigger condition: ___________________
   - Human owner: _______________________
   - SLA: _______________________________

6. Success metric measured at day 14:
   ____________________________________________

If question 4 has no items — the agent is read-only — your architecture is dramatically simpler. Start there.

Micro-case (what changes after 7-14 days)

A 60-person professional-services firm wires up an agent over a long weekend using a hosted orchestrator, direct CRM lookups, and a Slack escalation channel. By day 7, the agent is handling around 40-50 routine client-status queries per day, with about 12% escalating to a human. By day 14, escalations drop to 6%, the team has clawed back roughly 15 hours of senior-staff time per week, and the founder has stopped reading the Slack escalation channel because it's quiet. No DevOps was hired. No vector database was deployed. Total infrastructure cost lands under €400/month.

Note on this case: This example is illustrative — based on typical patterns we observe with companies of 30-500 employees, not a single named client. Specific numbers are rounded approximations of common ranges, not guarantees.

Tool tip (Course for Business): The single biggest predictor of whether a 30-person team's agent architecture survives month 3 is whether the people using it understand the underlying loop — context in, model decision, tool call, output, log. We teach this in the 6-week program with a Shoulder-to-Shoulder hot-seat session where each team builds and debugs their own agent live. Architecture that nobody on the team can debug is architecture that gets shut off the first time it misfires.

FAQ

Do we need a vector database? Probably not in the first quarter. If your agent's job is "look up structured data and act on it," a direct database query is faster, cheaper, and more accurate than embeddings. Add a vector store only when you have a clear use case for semantic search over unstructured text.

Hosted vs self-hosted LLM? For 30-500-employee SMBs, hosted is almost always the right call. Self-hosted models look cheaper on paper but Gartner's 1,000% miscalc finding applies — GPU costs, ops overhead, and version-upgrade work eat the savings. Revisit when you're past €15k/month in API spend.

Can one person own the whole architecture? Yes, if the architecture is small enough. That's the whole point of the four-component model. A capable ops lead with one trained champion can run this. If you need more than two people to keep the agent alive, the architecture is too complex for your stage.

What about security and EU AI Act compliance? For most SMB agents (internal productivity, customer-service drafts, internal data lookup), you fall well outside the EU AI Act's high-risk category. The fines apply to high-risk systems — biometric ID, credit scoring, employment decisions. For a logistics-status agent or invoice-lookup agent, standard data-handling practice is enough. But document your data flows now — auditors will ask later.

Bottom line

A 30-person team does not need enterprise architecture. It needs four components, one champion, a clear escalation path, and visible logs. Skip the vector DB, skip the self-hosted LLM, skip the orchestration framework with 800 GitHub stars. Ship the simplest thing that works, watch the logs, and let the agent earn its complexity.

Next step: write your decision template above. If you cannot fill it in, no architecture will save you.

If you want every employee to ship their first AI automation in five days — book a 30-min call and we'll map your team's first week: https://course.aiadvisoryboard.me/business