How to Run an AI Skill-Gap Assessment Without Hiring a Consultant

If you're an owner of a 60-200 person SMB and a vendor just quoted you €18,000 for an "AI readiness assessment" — pause. You can run a defensible one yourself in a week, and the output will be more useful because your own people built it.

Why do most skill-gap reports go in a drawer?

Because they grade the people instead of grading the gap.

A consultant report says "42% of your staff scored beginner on prompt engineering." Your COO reads it, nods, and files it. Nothing changes because nothing is actionable. There's no role attached to the gap, no workflow it blocks, no manager who owns closing it.

Definition: Skill-gap assessment — a snapshot of (current capability) vs (capability required by the work people actually do), broken down by role, with a closing plan per gap. Not a personality test, not a maturity score.

The DIY version we'll walk through skips the maturity-model theatre and goes straight to: who needs what, by when, to do the actual work in front of them.

What does the one-week shape look like?

Day 1: define the roles you're assessing. Day 2: ship the survey. Day 3-4: practical tasks for a sample. Day 5: score, map, and write the three-priority output.

You need three artifacts before you start:

1. A list of role families in scope (typically 6-12 for a 100-person SMB — not 47 individual titles).
2. A rubric that maps role family to required AI capability level (we'll build this below).
3. A named owner per department who runs the practical and signs off on the heatmap.

Definition: Role family — a group of jobs that share the same daily AI surface area. "Account executive" and "SDR" might be one family; "controller" and "AP clerk" probably aren't.

What goes in the 15-minute survey?

Six question blocks. Resist the urge to add a seventh.

1. Tool usage today (which tools, how often, paid or free, sanctioned or shadow).
2. Tasks where AI is already helping (free text, capped at three examples).
3. Tasks they wish AI could help with (free text, capped at three).
4. Self-rated confidence on a five-point scale across four capabilities: writing prompts, judging output quality, redacting sensitive data, recovering from a bad output.
5. Training history (any structured training, hours, when, by whom).
6. One open question: "What's the one task that eats your week that you'd hand to AI tomorrow if you could?"

That last question is gold. The answers cluster into 4-6 patterns per department, and those patterns become your training priorities.

Definition: Shadow AI — sanctioned-or-not, paid-or-not AI tools that employees use on work tasks without IT's knowledge. Common pattern: every 50+ person SMB has 4-8 shadow tools running.

What does the three-task practical look like?

The survey tells you what people say. The practical tells you what they can do.

Pick three tasks per role family. They should be small (15-30 minutes), realistic, and graded by output, not by approach. Examples for an AE role family:

• Task 1: Draft a discovery-call follow-up email from a meeting transcript.
• Task 2: Spot one factual error and one tone problem in an AI-generated case study.
• Task 3: Redact a sensitive contract excerpt before pasting it into a public AI tool.

Sample 15-20% of each role family. Twelve people across a 100-person SMB is enough to triangulate the survey. Score against the rubric below.

Copy/paste rubric template

This is the rubric we hand teams as a starting point. Adjust the capability levels and required-by-role mapping to your business.

ROLE FAMILY: [e.g., Account Executive]
REQUIRED CAPABILITY LEVELS (target column):

Capability                | Required level (1-4)
--------------------------+----------------------
Tool fluency              | 2
Prompt design             | 3
Output judgment           | 3
Data-handling discipline  | 3
Workflow integration      | 2
Recovery from bad output  | 2

LEVEL DEFINITIONS:
1 = Aware     - knows the tool exists, has tried it once
2 = Functional - uses on routine tasks weekly, follows examples
3 = Fluent    - adapts prompts to context, judges quality, redacts data
4 = Coach     - teaches others, designs workflows, debugs failures

SCORING PRACTICAL (per task):
- Output quality: 0-3
- Approach soundness: 0-3
- Data handling: 0-3 (zero is automatic-fail at the role level)

GAP = required level - observed level (averaged)
Negative or zero = not a training target.
+1 = one cohort lab.
+2 or more = structured training program.

OUTPUT:
- Heatmap (role family x capability) with red/amber/green cells.
- Three priorities: largest gaps that block the most-named tasks from survey question 6.
- Named owner per priority.

The rubric is opinionated on purpose. A document that lets every reading land on "we need general AI training" is not a rubric — it's a horoscope.

Tool tip (Course for Business): When SMBs run this assessment themselves, the most common stall is at the rubric step — they over-design the levels and never ship. Our 6-week program brings a pre-built role rubric for the eight most common SMB role families (sales, CS, marketing, finance, ops, HR, eng, leadership) and an AI Champions (1:15-20) pod that runs the practical for you in week one. Augment, don't replace — the assessment surfaces the gap; your managers stay the ones deciding what to do about it. Walk through the program at https://course.aiadvisoryboard.me/business.

Team scan (what AI champions report after week 1)

• Survey response rate: aim for 70%+ in week one; below 50% means leadership hasn't endorsed it visibly.
• Most useful single signal: the open question about "the task that eats your week."
• Highest-gap capability across SMBs we see: data-handling discipline (redaction, what-to-paste, what-not-to-paste).
• Lowest-gap capability: tool fluency — most people have tried ChatGPT.
• Surprise pattern: senior people often score lower than juniors on output judgment because they delegate review.
• Shadow AI surfaces in question 1 — expect 4-8 unsanctioned tools per 50+ headcount.
• One AI champion per ~17 staff runs the practical scoring; a manager owns the rubric calibration.
• First friction: people fear the practical is a performance test — frame it as "we're testing the curriculum, not you."
• First win: department heads recognize the heatmap pattern within minutes of seeing it.
• Saved consultancy spend: typically €12-25K vs an external assessment, repeatable quarterly.

Micro-case (what changes after 7-14 days)

A 120-person professional services firm ran this in five working days. Forty-two people filled the survey; twelve did the practical. The heatmap showed three obvious red zones: data-handling discipline across the whole firm, output judgment in the junior consulting tier, and workflow integration in marketing. The leadership team had been about to spend €22K on a generic "AI 101" program for everyone. After seeing the heatmap, they redirected the budget into one targeted data-handling workshop for the whole firm, a four-week judgment-focused lab for junior consultants, and a marketing-specific workflow build with a champion. Six weeks later, override rates on AI-drafted client work had dropped roughly 40%, and the junior consultants were running their own peer reviews of AI output without manager prompting.

Note on this case: This example is illustrative — based on typical patterns we observe with companies of 30-500 employees, not a single named client. Specific numbers are rounded approximations of common ranges, not guarantees.

Tool tip (Course for Business): The trap with DIY assessments is that they get done once, the heatmap is shared, and then nothing happens because no one owns closing the gaps. The Shoulder-to-Shoulder hot seat in our 6-week program is built around that exact failure — a champion sits with each red-zone team for a week, ships the first real workflow with them, and hands over a maintenance rhythm. The assessment is one Monday; the close is what the next six weeks are for. Book a 30-min mapping call at https://course.aiadvisoryboard.me/business.

FAQ

Can we just use the survey and skip the practical? You can, and you'll get a usable directional read in two days. But the survey alone overstates capability — people self-rate generously. The practical is what separates "they say they can redact contracts" from "they actually can." If you skip it, run a one-week pilot of the top priority instead and treat that as your reality check.

What if our role families are unusual? Build them. The rubric is opinionated on the levels, not on the role mix. A construction firm we worked with had "site superintendent" as a role family with very different required levels than office staff — the heatmap still worked because the capability columns are stable across industries.

How often should we re-run this? Once at baseline, then every quarter for the first year, then twice a year. Capabilities move faster than annual cycles, and the heatmap drift between quarters is itself a useful signal — if nothing has moved in 90 days, your training isn't sticking.

Won't anonymous surveys give us better data? Slightly higher response rates, materially worse follow-up. If you can't attach a name to a gap, you can't assign an owner to close it. Signed responses, with leadership saying "this is for designing training, not evaluating people" — that's the framing that works.

Conclusion

A consultant gives you a binder. The DIY assessment gives you three priorities and three owners. The owners are the difference.

Block one week. Define your role families on Day 1. Ship the survey by end of Day 2. Score the practical by Day 4. Show the heatmap and pick three priorities by end of Day 5.

If you want every employee to ship their first AI automation in five days — book a 30-min call and we'll map your team's first week at https://course.aiadvisoryboard.me/business.